Requirements

Imagine you are building a simplified, in-memory version of Kafka. This system should allow Producers to publish messages to Topics, and Consumers to subscribe to those Topics and receive those messages. We are NOT building a distributed system here. The focus is on the **core messaging and subscription mechanisms**, emphasizing clean object-oriented design principles. We will leave out features like persistence, replication, and partitioning for simplicity. Think of it this way: We need a "message bus" within a single JVM. This message bus must reliably deliver messages to all subscribed consumers. This reliability, in our simplified context, means ensuring that all active consumers receive each message exactly once (within the scope of a single JVM). This is a significantly simplified view of real-world Kafka, but a useful exercise for exploring design patterns. # Requirements ## Functional Requirements 1. **Topic Creation:** The system must allow the creation of new Topics. A Topic is a named channel to which producers can publish messages, and from which consumers can receive messages. 2. **Message Publishing:** Producers should be able to publish messages to a specific Topic. Each message consists of a key and a value, both strings. 3. **Consumer Subscription:** Consumers should be able to subscribe to one or more Topics. 4. **Message Delivery:** When a message is published to a Topic, all consumers subscribed to that Topic should receive the message. Each active consumer must receive the message exactly once. 5. **Consumer Unsubscription:** Consumers should be able to unsubscribe from a Topic. After unsubscribing, a consumer should no longer receive messages published to that Topic. 6. **Message Filtering (Optional):** Consumers should be able to specify a filter criteria (e.g., based on message key or value) when subscribing to a Topic. Only messages matching the filter criteria will be delivered to the consumer. This allows implementing basic content-based routing. 7. **Offset Management (Simplified):** Consumers should maintain their own offset per topic. Each consumer should resume consuming messages from the last known offset. ## Non-Functional Requirements 1. **Thread Safety:** The system must be thread-safe. Multiple producers and consumers may operate concurrently. Protect shared state using appropriate synchronization mechanisms. 2. **Extensibility:** The design should be extensible to support new message formats, filtering mechanisms, and delivery strategies in the future. 3. **Modularity:** The system should be designed with clear separation of concerns. Components like message handling, subscription management, and delivery should be modular and loosely coupled. 4. **Testability:** The code should be easily testable. Design interfaces and classes to facilitate unit and integration testing. Favor dependency injection to allow mocking of dependencies. 5. **Performance:** While raw speed is not the primary focus, the system should avoid unnecessary bottlenecks. Consider the impact of concurrency and synchronization on performance. 6. **Error Handling:** The system should handle errors gracefully. Consider what happens when a consumer fails to process a message. Should the message be retried? Should the consumer be notified? In our simplified system, we can focus on logging exceptions. ## Core Entities 1. **Topic:** Represents a named channel for message publication and subscription. Manages the list of subscribers and the sequence of messages. 2. **Message:** Represents the data being published and consumed. Contains a key and a value. Potentially extendable to include metadata (e.g., timestamp). 3. **Producer:** An entity that publishes messages to a Topic. Encapsulates the logic for sending messages. 4. **Consumer:** An entity that subscribes to a Topic and receives messages. Provides a callback interface for processing received messages. 5. **Subscription:** Represents the association between a Consumer and a Topic, including any filtering criteria and the consumer's offset. 6. **Broker:** The central component that manages Topics, Producers, and Consumers. It acts as the intermediary for message delivery. It encapsulates the core logic of the system. 7. **Offset:** Represents the current position of a consumer in a topic's message stream.

Implementation of Kafka.

Requirements

Interview Simulation

Functional Requirements

💡 Interview Tip

Non-Functional Requirements

Core Entity Identification

Premium Content