Using Camel-cassandraql on a Kubernetes Cassandra cluster with Fabric8
14 Oct 2016In the last months I blogged about Testing Camel-cassandraql on a Dockerized cluster. This time I’d like to show you a similar example running on Kubernetes. I’ve worked on a fabric8 quickstart splitted in two different projects: Cassandra server and Cassandra client. The server will spin up a Cassandra cluster with three nodes and the client will run a camel-cassandraql route querying the keyspace in the Cluster. You’ll need a running Kubernetes cluster to follow this example. I decided to use Fabric8 on Minikube, but you can also try to run the example using Fabric8 on Minishift. Once you have your environment is up and running you can start to execute the following sections commands.
Spinning up an Apache Cassandra cluster on Kubernetes
For this post we will use Kubernetes to spin up our Apache Cassandra Cluster. To start the cluster you can use the Fabric8 IPaas Cassandra app. Since we are using Minikube, you’ll need to follow the related instructions.
This project will spin up a Cassandra Cluster with three nodes.
As first step, in the cassandra folder, we will need to run
mvn clean install
and
mvn fabric8:deploy
Once we’re done with this commands we can watch the Kubernetes pods state for our specific app and after a while we should be able to see our pods running:
oscerd@localhost:~/workspace/fabric8-universe/fabric8-ipaas/cassandra$ kubectl get pods -l app=cassandra --watch
NAME READY STATUS RESTARTS AGE
cassandra-2459930792-lfh93 1/1 Running 0 15s
cassandra-2459930792-m8c9p 1/1 Running 0 15s
cassandra-2459930792-zxiv3 1/1 Running 0 15s
I usually prefer the command line, but you can also watch the status of your pods on the wonderful Fabric8 Console and with Minikube/Minishift is super easy to access the console, simply run the following command:
minikube service fabric8
or on Minishift
minishift service fabric8
In Apache Cassandra nodetool is one of the most important command to manage a Cluster and executing command on single node.
It is interesting to look at the way the Kubernetes team developed the Seeds discovery into them Cassandra example. The class to look at is KubernetesSeedProvider and it implements the SeedProvider interface from Apache Cassandra project. The getSeeds method is implemented here: if the KubernetesSeedProvider is unable to find seeds it will fall back to createDefaultSeeds method that is exactly the simpleSeedProvider implementation from Apache Cassandra Project.
The Kubenertes-cassandra jar is used in the Cassandra DockerFile from Google example and it is added to the classpath in the run.sh script.
In the Cassandra fabric8-ipaas project we used the deployment.yml and service.yml configuration. You can find the details in the repository. You can also scale up/down your cluster with this simple command:
kubectl scale --replicas=2 deployment/cassandra
and after a while you’ll see the scaled deployment:
oscerd@localhost:~/workspace/fabric8-universe/fabric8-ipaas/cassandrakubectl get pods -l app=cassandra --watch
NAME READY STATUS RESTARTS AGE
cassandra-2459930792-lfh93 1/1 Running 0 28m
cassandra-2459930792-m8c9p 1/1 Running 0 28m
Our cluster is up and running now and we can run our camel-cassandraql route from Cassandra client.
Running the Cassandra-client Fabric8 Quickstart
In Fabric8 Quickstarts organization I’ve added a little Cassandra-client example. The route will do a simple query on the keyspace we’ve just created on our Cluster. This quickstart requires Camel 2.18.0 release to work fine, now that 2.18.0 is out we can cover the example. The quickstart will first execute a bean to pre-populate the Cassandra keyspace as init-method and it defines a main route. It’s a very simple quickstart (single query/print result set), but it shows how it is easy to interact with the Cassandra Cluster. You don’t have to add contact points of single nodes to your Cassandra endpoint URI or use a complex configuration, you can simply use the service name (“cassandra”) and Kubernetes will do the work for you.
You can run the example with:
mvn clean install
mvn fabric8:run
You should be able to see your cassandra-client running:
oscerd@localhost:~/workspace/fabric8-universe/fabric8-quickstarts/cassandra-client$ kubectl get pods
NAME READY STATUS RESTARTS AGE
cassandra-2459930792-lfh93 1/1 Running 0 45m
cassandra-2459930792-m8c9p 1/1 Running 0 45m
cassandra-2459930792-ms265 1/1 Running 0 15m
cassandra-client-3789635050-e1f2g 1/1 Running 0 1m
With the fabric8:run command you’ll see the logs of the application in the console directly:
2016-10-14 09:06:15,786 [main ] INFO DCAwareRoundRobinPolicy - Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
2016-10-14 09:06:15,789 [main ] INFO Cluster - New Cassandra host cassandra/10.0.0.67:9042 added
2016-10-14 09:06:15,790 [main ] INFO Cluster - New Cassandra host /172.17.0.10:9042 added
2016-10-14 09:06:15,790 [main ] INFO Cluster - New Cassandra host /172.17.0.12:9042 added
2016-10-14 09:06:20,701 [main ] INFO SpringCamelContext - Apache Camel 2.18.0 (CamelContext: camel-1) is starting
2016-10-14 09:06:20,703 [main ] INFO ManagedManagementStrategy - JMX is enabled
2016-10-14 09:06:20,928 [main ] INFO DefaultTypeConverter - Loaded 190 type converters
2016-10-14 09:06:20,977 [main ] INFO DefaultRuntimeEndpointRegistry - Runtime endpoint registry is in extended mode gathering usage statistics of all incoming and outgoing endpoints (cache limit: 1000)
2016-10-14 09:06:21,228 [main ] INFO SpringCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
2016-10-14 09:06:21,231 [main ] INFO ClockFactory - Using native clock to generate timestamps.
2016-10-14 09:06:21,503 [main ] INFO DCAwareRoundRobinPolicy - Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
2016-10-14 09:06:21,548 [main ] INFO Cluster - New Cassandra host cassandra/10.0.0.67:9042 added
2016-10-14 09:06:21,548 [main ] INFO Cluster - New Cassandra host /172.17.0.11:9042 added
2016-10-14 09:06:21,548 [main ] INFO Cluster - New Cassandra host /172.17.0.10:9042 added
2016-10-14 09:06:22,047 [main ] INFO SpringCamelContext - Route: cassandra-route started and consuming from: timer://foo?period=5000
2016-10-14 09:06:22,075 [main ] INFO SpringCamelContext - Total 1 routes, of which 1 are started.
2016-10-14 09:06:22,083 [main ] INFO SpringCamelContext - Apache Camel 2.18.0 (CamelContext: camel-1) started in 1.374 seconds
2016-10-14 09:06:23,133 [0 - timer://foo] INFO cassandra-route - Query result set [Row[1, oscerd]]
2016-10-14 09:06:28,092 [0 - timer://foo] INFO cassandra-route - Query result set [Row[1, oscerd]]
As you can see everything is straightforward and you can think about more complex architectures. For example: an Infinispan cache with a persisted cache store based on Apache Cassandra Cluster with Java application, Camel routes or Spring-boot applications interacting with the Infinispan platform.
The route of the example is super simple:
<camelContext xmlns="http://camel.apache.org/schema/spring" depends-on="populate">
<route id="cassandra-route">
<from uri="timer:foo?period=5000"/>
<to uri="cql://cassandra/test?cql=select * from users;&consistencyLevel=quorum" />
<log message="Query result set ${body}"/>
</route>
</camelContext>
and it abstracts completely the details of the Cassandra cluster (how many nodes? Where are the nodes? What are the addresses of them? and so on).
Conclusions
This example shows only a bit of the power of Kubernetes/Openshift/Fabric8/Microservices world and in the future I would like to create a complex architecture quickstart. Stay tuned!