Adding Persistence with MongoDB Atlas

Introduction

When building a real-time chat application, one of the key features users expect is the ability to view past conversations. Persisting chat messages ensures that users can reconnect and access their conversation history seamlessly. This functionality not only improves user experience but also opens doors to additional features like search and analytics.

In this blog, we’ll discuss why we chose to persist chat messages and why MongoDB Atlas was our database of choice. We’ll also explore the implementation details, challenges faced, and lessons learned during the process.

Why Persist Chat Messages?

1. User Experience: Without persistence, users would lose all message history upon disconnecting or refreshing the app. By storing messages, users can pick up conversations where they left off.
2. Scalability for Features: Persisted data allows for advanced features like search, chat analytics, and integration with other systems.

Why MongoDB Atlas?

Choosing the right database was crucial for a project like this. We opted for MongoDB Atlas for several reasons:

• Simplicity: MongoDB Atlas provides a fully managed cloud-based database, eliminating the need for manual setup and maintenance.
• Scalability: With support for horizontal scaling through sharding, MongoDB Atlas can grow alongside our application.
• Flexibility: The document-oriented structure of MongoDB allows for easy storage of JSON-like data, making it a perfect fit for chat messages.
• Global Distribution: Atlas offers multi-region deployment, reducing latency for users around the world.

---

Implementation

1. Setting up MongoDB Atlas

To get started with MongoDB Atlas:

1. Create a Cluster: Sign up at MongoDB Atlas and create a cluster. Choose the free tier for testing or a paid tier for production.
2. Configure Network Access: Add IP addresses that can access the database, or allow access from all IPs (not recommended for production).
3. Generate Credentials: Create a database user with a secure password and note the connection string.

2. Connecting to the Database

Using Go’s MongoDB driver, we established a connection to the Atlas cluster:

package main

import (
    "context"
    "log"
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
)

func main() {
    // MongoDB connection string
    uri := "your-mongodb-connection-string"

    // Create a new client
    client, err := mongo.Connect(context.TODO(), options.Client().ApplyURI(uri))
    if err != nil {
        log.Fatal(err)
    }
    defer client.Disconnect(context.TODO())

    log.Println("Connected to MongoDB Atlas!")
}

3. Storing Chat Messages

The messages collection in our database stores each chat message as a document. A typical schema looks like this:

{
  "room_id": "12345",
  "timestamp": "2024-12-12T10:30:00Z",
  "sender": "john_doe",
  "content": "Hello, World!"
}

To save messages:

collection := client.Database("chatDB").Collection("messages")
message := bson.M{
    "room_id":   "12345",
    "timestamp": time.Now(),
    "sender":    "john_doe",
    "content":   "Hello, World!",
}
_, err := collection.InsertOne(context.TODO(), message)
if err != nil {
    log.Fatal(err)
}
log.Println("Message saved successfully!")

4. Querying Messages

When a user joins a chat room, we retrieve past messages based on the room_id:

filter := bson.M{"room_id": "12345"}
cursor, err := collection.Find(context.TODO(), filter)
if err != nil {
    log.Fatal(err)
}
defer cursor.Close(context.TODO())

var messages []bson.M
if err = cursor.All(context.TODO(), &messages); err != nil {
    log.Fatal(err)
}
log.Println("Messages retrieved: ", messages)

---

Challenges Faced

1. Structuring the Data Schema for Scalability

Initially, we considered embedding all messages for a room in a single document. However, this approach would make the document grow indefinitely, leading to performance issues. Instead, we adopted a flat structure where each message is a separate document indexed by room_id.

2. Managing Database Connections

Handling a large number of concurrent connections was a concern. We used connection pooling, which is supported natively by MongoDB’s Go driver, to efficiently manage connections and reduce overhead.

---

Lessons Learned

1. Designing a Schema with Future Features in Mind

A well-thought-out schema can save significant effort down the line. For example, including a timestamp field enabled us to sort messages easily, and adding room_id allowed us to shard the database if needed.

2. Integrating Cloud Databases into a Go Application

MongoDB Atlas’s simplicity and robust documentation made integration straightforward. However, we learned the importance of securely managing credentials, using environment variables to avoid hardcoding sensitive information.

---

Conclusion

Adding persistence to our chat application using MongoDB Atlas has significantly enhanced its functionality and scalability. Users can now access their chat histories, and the app is better prepared for future growth. In our next blog, we’ll discuss how to scale and test this real-time chat application to handle millions of users effectively.

Stay tuned!