Saturday, 24 September 2022

Cassandra internal architecture

Apache Cassandra is a NoSQL distributed database which can handle large amounts of data across multiple commodity servers to support highly available system in a distributed environment with no Single point of failure.

In this blog I will try to explain Cassandra basic architecture and working. I will also try to explain why it was designed in such a way and what are few best fit application usage examples.

Sunday, 21 August 2022

Master-master vs master-slave database architecture

In this blog we will understand single copy, master-slave and multi-master database architecture. We are going to understand various pros and cons of each architecture with some examples.

Database without Replication (Single Copy)

In this architecture one standalone database server is used for all read and write DB operations from the application.

Saturday, 11 September 2021

Design and Implement Rate Limiter

Rate limiter is one of the most important aspects when designing a microservice system. It is used to limit the usage of an api upto a particular threshold. When an api exceeds its TPS (Transactions Per Second) threshold limit, it should ignore those extra requests to save all downstream services and should send proper http status to its client.

Let's take an example to understand this. Let's say there is an api whose maximum TPS it can handle is 500 requests per second. If we apply a rate limiter on this api upto 500 request per second and we experience 700 request per second in a particular time window, then 200 extra requests will be ignored with an http status 429 i.e. TOO_MANY_REQUEST. Before going into detail of how to implement a Rate limiter in a distributed environment, let's understand various advantages of using it.

Tuesday, 20 July 2021

Streaming NRT data with kafka connect and Debezium

Event driven architecture principle is very popular for past few years in microservice systems. Providing Near Real Time (NRT) data to other services as soon as possible is core problem for all organisations. 
This is also known as Change data capture Design pattern where capturing each database change event, then wrapping it and pushing it to other pipeline or services. 
This is Duality of Streams and Tables where database table change events are pushed over a stream to other systems by an order of timestamp just to make a snapshot of database at other systems. 

Wednesday, 27 January 2021

Useful kafka commands

For past few months I have been working with many kafka production issues and I am still trying to learn many new things about it. But while working on any kafka issue, you always want some commands to be very handy and just one click away in a file or in a blog.

Here are few kafka commands that may be very helpful to you while learning or solving any of your kafka issues. 

Here I would be referring KAFKA_HOME as your path to kafka home.