System Design – Post Live Comments

Designing a live comment system involves several components: the client-side interface, the WebSocket server, session server, and often a backend for persisting comments.

System Requirements

1. able to have user comments for a post
2. able to push latest comments to connected viwers
3. able to retrieve historical comments
4. all comments will be persistent.
5. Scalable, with high availability, and fault-tolerance

Scale Estimation

For a platform with users about 1 billion, and DAU 100 million, and 20% of them are following comments concurrently, that’s about 20M connections and posts.
Assume each posts have average 1 new comments per second, that’s about 20M new comments per second in total.
Assume each comment has an average length of 40 words, about 200 chars, in total, daily comments takes about 0.2kb * 20M = 4GB/s.

Below is a high-level design outline:

1. System Architecture Overview:

a. Client-Side (Web Application)

• User Interface: A dynamic webpage where users can view and post comments in real time.
• WebSocket Client: JavaScript code to manage WebSocket connections for sending and receiving messages.

b. WebSocket Server
Manages real-time, bi-directional communication with clients. Utilizes session information for personalized communication.

• Connection Management: Handles incoming WebSocket connections from multiple clients.
• Message Handling: Receives messages (new comments) from clients, broadcasts them to other connected clients, and optionally sends them to the backend server for storage.

c. Backend Server

• Database: Stores comments for historical reference and loads previous comments when a new client connects.
• API: Provides endpoints for fetching historical comments and receiving new comments from the WebSocket server.

d. Session Server:

Incorporating a session server into a WebSocket-based architecture is a strategy to manage user sessions and ensure consistent communication with clients. The session server typically handles user authentication, session management, and potentially load balancing.

• Handles user authentication and session creation.
• Generates a unique session identifier for each client.
• After successful authentication and session creation, the client’s connection is upgraded to a WebSocket connection.
• May assign the client to a specific WebSocket server based on load balancing.

e. Message Queue

The message queue acts as a buffer and a mechanism for rate control, ensuring the WebSocket server can handle and distribute messages efficiently without being overwhelmed.

2. Client-Side Implementation:

• HTML & CSS for the comment section layout.
• JavaScript:
• Create a WebSocket connection to the server.
• Send new comments to the server via the WebSocket.
• Update the comment section in real-time when new comments are received from the server.
• (Optionally) Fetch historical comments from the backend API on page load.

3. WebSocket Server Implementation:

• Use a WebSocket library (like ws for Node.js).
• Manage connections from multiple clients.
• Broadcast incoming comments to all connected clients.
• (Optionally) Forward comments to the backend server for storage.

4. Backend Server and Database (Optional):

• REST API: To handle requests for historical comments.
• Database: Store and retrieve comments. Could be SQL (e.g., PostgreSQL) or NoSQL (e.g., MongoDB).

5. Data Flow:

User Posts a Comment:

• The client sends the comment through the WebSocket connection.

WebSocket Server Receives the Comment:

• Broadcasts the comment to all connected clients.
• (Optionally) Sends the comment to the backend server for storage.

Other Clients Receive the Comment:

The comment appears in real-time in their comment sections.

Loading Historical Comments:

• When a new client connects, it optionally fetches past comments from the backend server.

6. Security and Performance Considerations:

• Authentication: Ensure that only authenticated users can post comments.
• Rate Limiting: To prevent spam and system overload.
• Cross-Site WebSocket Hijacking Prevention: Verify the origin of WebSocket requests.
• Scalability: The server should be scalable to handle many concurrent connections.
• Data Validation: Validate comments to prevent injections and XSS attacks.

Periodical Pull (Polling):

Pros:

1. Simplicity: Easier to implement and understand, especially in environments with limited support for WebSockets.
2. Compatibility: Works with all web browsers and does not require special server configurations.
3. Predictable Server Load: Requests are made at regular intervals, allowing for predictable server load.

Cons:

1. Latency: There’s a delay between new comments being posted and them appearing to other users, as updates only happen at each poll interval.
2. Inefficient: The server is hit with requests at regular intervals, regardless of whether there’s new data or not, leading to unnecessary load and traffic.
3. Not Real-Time: Doesn’t provide a truly real-time experience, which might be crucial for a live commenting system.

Push (WebSockets):

Pros:

1. Real-Time: Delivers an immediate, real-time experience, which is essential for a live commenting system.
2. Efficient: The server only sends data when there’s new information, reducing unnecessary load and network traffic.
3. Bi-Directional Communication: Allows for full-duplex communication, enabling more interactive features.

Cons:

1. Complexity: More complex to implement and manage, especially in terms of handling connections and ensuring scalability.
2. Browser and Server Support: While widely supported, some older browsers or servers might not fully support WebSockets.
3. Potential for Increased Server Load: Depending on the use case, maintaining multiple open WebSocket connections can be resource-intensive for the server.

Conclusion for a Live Comment System:

• Push (WebSockets) is generally better for a live comment system because:
  • It provides a real-time experience, crucial for interactive commenting.
  • It’s more efficient in terms of network usage as data is only transmitted when there’s new content.
  • Offers a more engaging and responsive user experience.
• Consider Periodical Pull (Polling) if:
  • Real-time updates are not critical.
  • There are limitations in terms of technology stack or compatibility requirements.
  • The system is small-scale or used in a context where server resources are constrained.

In summary, for a live comment system where immediate feedback and interactivity are vital, a push approach using WebSockets is typically more suitable. However, the final decision should also consider the specific requirements and constraints of the project.

Related technology

Server-Sent Events (SSE) is a technology that allows a server to send updates to clients (like web browsers) in real-time. It’s a part of the HTML5 specification and is commonly used in applications where the server needs to push updates to the client without the client specifically requesting it.

Key Characteristics of SSE with Multiple Clients:

1. One-Way Communication: SSE is designed for unidirectional communication from the server to the client. This means the server can push data to each connected client, but the clients cannot send data back to the server through the same connection.
2. Multiple Concurrent Connections: A server can handle numerous concurrent SSE connections. Each client that connects to the server for SSE has its own unique connection.
3. Scalability Considerations: While SSE can support multiple clients, the number of concurrent connections a server can handle efficiently depends on its resources and configuration. Handling a large number of connections may require a well-designed infrastructure, possibly involving load balancers and a distributed architecture.
4. Use of HTTP: SSE works over standard HTTP, making it compatible with most web infrastructures and easy to implement. Each client connection to the server is a standard HTTP connection.
5. Stateless Nature of HTTP: Despite being a stateless protocol, HTTP can be utilized to maintain SSE connections. The server keeps track of each connection and its state to ensure continuous data flow.
6. Client Reconnection Mechanism: SSE has a built-in reconnection feature. If a connection is lost, the client automatically attempts to reconnect to the server, making it robust for real-time applications.
7. Optimization for Text Data: SSE is optimized for sending text data (like JSON, XML, plain text). It’s particularly well-suited for scenarios where the server needs to send updates or notifications to clients.

WebSockets is a communication protocol that provides a full-duplex communication channel over a single, long-lived connection. It is commonly used for real-time web applications because it allows for interactive communication between a client (like a web browser) and a server.

Some Other Parts to consider:

1. Performance consideration on the client side
2. Offline support, pagination, reconnection support(always keep at most one connection for new posts)
3. Monitoring and logging
4. Security and Compliance
5. Operational Benefits

Questions

Q: how to scaling WebSocket Servers
Answer:

1. Horizontal Scaling: Add more server instances to distribute load.
   Load Balancing: Use a load balancer to distribute connections across servers.
2. Stateless Design: Make servers stateless for flexibility in handling connections.
3. Session Stickiness: For stateful connections, ensure clients consistently connect to the same server.
4. Distributed Cache/Data Store: Use for storing session states and user data accessible by any server.
5. Microservices Architecture: Break down functionalities for independent scaling.

Q: How to handling WebSocket Server Downtime
Answer:
1.Redundancy: Deploy servers across multiple locations for backup.

2. Failover Mechanisms: Implement automatic redirection to healthy servers during failures.
3. Health Checks and Monitoring: Continuously monitor server health to detect and address issues quickly.
4. Graceful Degradation: Design client applications to manage server downtimes without major disruptions.
5. Backup Systems: Have simpler backup systems for basic functionality during outages.
6. Disaster Recovery Plan: Prepare for major incidents with data backups and service restoration strategies.

Ref:

1. https://systemdesign.one/live-comment-system-design/
2. https://www.jb51.net/article/283610.htm
3. https://www.geeksforgeeks.org/design-facebooks-live-update-of-comments-on-posts-system-design/