Tech Bites

For a distributed system to be consistent, a voting mech. is used, 'Quorum' : minimum no of nodes that must agree to make an operation valid. Think: 5 replicas, need 3 votes to write/read. Why? So reads & writes overlap, ensuring you always see recent data. Math: R+W>N = consistency

Emergent leadership refers to a pattern in distributed system where one node dynamically becomes coordinator through an algorithmic process (raft, paxos, etc) rather than pre-designated. Advantages: No single point of failure, automatic failover, highly available

High watermark is a highest offset that's been safely replicated, ensuring you never read data that could vanish in a failure. It is used to ensure that all nodes (primary or replica) have seen all the messages up to that point. It is really a unsung hero of distributed chaos.

In Distributed Systems, Nodes' clocks are never perfectly synced. Before committing, a node waits a few ms to ensure all nodes—even those with slower clocks—have passed its timestamp. This prevents read anomalies.

A vector database is a specialized type of database designed to store and search through 'vectors' - which are essentially lists of numbers that represent the meaning or characteristics of data like text, images, or audio. A vector database stores these numerical fingerprints and can quickly find items with similar fingerprints.

ZooKeeper is a centralized service for maintaining config info, providing group services, and coordinating highly reliable distributed system. Consensus, group management, and presence protocols will be implemented by the service so that the applications do not need to implement them on their own.

Merkle Tree is a tree-like data structure whose leaf node contains hash of data and non-leaf node contains hash of its child nodes. Merkle tree is widely used in distributed systems to compare data b/w two nodes efficiently, if in case replication is required.

In distrubuted systems, nodes periodically communicates with each other and exchange their state information. It is used for fauly detection, achieve consensus, maintaining node membership list, etc.

RPC stands for Remote Procedure Call, a communication protocol that allows a program to execute code or call functions on a remote system(another node in a distributed system) as if they were local function call. For ex, Uber's ride request service calls the driver location service via RPC to find nearby drivers. Known RPC are gRPC, Apache Thrift, etc

Vector clocks are a way to track the causal order of events in a distributed system. They are used to detect conflicts in distributed systems. By comparing vector clocks, the system can identify if an event on one node causally happened before, after, or concurrently with an event on another node, enabling effective conflict resolution and ensuring consistency.

Sloppy Quorum allows read and write operations to proceed even if some nodes are unreachable, prioritizing availability over immediate consistency. Hinted Handoff temporarily stores data on a reachable node when the intended node is down, ensuring data is eventually transferred to the correct node once it’s back online.

In distributed systems, you can't have exactly once delivery. You can have at most once delivery or at least once delivery with weighing more towards at least once delivery.This is because of the CAP theorem.

Also knowns as Robots.txt, Standard used by websites to communicate with web crawlers. A file that tells web crawlers which pages they can and cannot access on a website.

In distributed systems, you can only guarantee two out of three: Consistency, Availability, and Partition Tolerance. Understanding this trade-off is crucial for designing distributed systems.

A space-efficient probabilistic data structure that tells you whether an element is definitely not in a set or probably in a set. Perfect for quick lookups when false positives are acceptable but false negatives are not.