Go + MongoDB

Published 05-13-2019 23:10:42

It's no secret that I'm in love with Go. It's also no secret that I really, really prefer MongoDB over most other datastores, with the exception of some key/value stores like bboltdb. This is by no means a sales pitch for any of these. At the end of the day, know your tools well and get the job done. I will work with anything that makes sense for the situation. Now… with that said, Go and MongoDB are my goto (no pun intended) for prototyping as well as “get s**t done” (within reason).

In the beginning…

I did what most Go devs have done and started with mgo. I've been using it for several years and it's been just fine. My only real issues were with handling multiple connections. If you created a copy of the main session and “established a connection”, any childSession.Close() on the children/copies doesn't actually close until the main session.Close() is called. This has then lead to too many files open on the server and just completely killing it. { “insert”: “sad trombone” } So… instead of having one main connection, I would create new connections for every transaction and defer conn.Close(). This seemed to work just fine with minimal overhead. I even asked the fantastic @skriptable the best practice last year (2018) when attending MongoDB World. There really seemed to be no real consensus other than whatever worked at the time. No worries. The production systems are peachy, life goes on.

A new contender has entered the ring

In comes 15/03/2019 and the v1.0.0 of the offical MongoDB Go driver is released. I'd been watching the betas as they progressed, not committing to it as I've got tons of production code using the mgo driver. Now that the driver is out and we should be shifting over to official code… I've got a “prod crunch” in progress. Yeah, I'm not swapping out and starting fresh at this point. Thankfully, we had a hard stop on April 4th with nothing really lined up right away. Got a bit of down time, plus some tech-debt that needs some love. Time to take the opportunity to test, abuse, and use it in production ready code. I have to say, I'm quite happy with it. Over all, I think this driver encourages cleaner, more expressive code. The one I'm super happy about is the ability to “stream” the results to a writer using an iterator rather than having to load everything to memory and then push to your writer. The “load to memory” has forced me to write creative code / queries in the past that I just preferred not to. It should really be up to the client / caller in my NAAH (not-at-all-humble) opinion how they get their data back, not in some arbitrary chunking / pagination that I impose (within reason, of course).

It's just that one thing

Ok… 2. First, if you want to use transactions, they're only applicable to replica sets. This means that code has to be aware of what's it's being deployed against or more code has to be written up front in order to handle both instances, likely toggling with a flag at boot. Ideally, this should be transparent to the client. But, I think I can live with this for the moment. We shall see with time.

Second, you can reuse the same client. It would appear that if you close a connection, your client must be disposed of completely. Take the following code as an example:


// PLEASE SECURE YOUR DB PROPERLY
// this is for demo purposes only
client, err := mongo.NewClient(context.TODO(), options.Client().ApplyURI("mongodb://localhost:27017"))
if err != nil {
    panic(err) // handle this properly - this is for demo purposes only
}

if err := client.Ping(context.TODO(), readpref.Primary()); err != nil {
    panic(err) // handle this properly - this is for demo purposes only
}

// now... DON'T DO THIS
if err := client.Disconnect(context.TODO()); err != nil {
    panic(err) // handle this properly - this is for demo purposes only
}

// attempt to "reconnect" to the client
if err := client.Connect(context.TODO()); err != nil {
    panic(err) // handle this properly - this is for demo purposes only
}

// this will eventually timeout
if err := client.Ping(context.TODO(), readpref.Primary()); err != nil {
    panic(err) // handle this properly - this is for demo purposes only
}

Now, the above example is a little contrived but it does serve to illustrate the point. The client.Connect(…) does not error nor does it actually connect. The additional client.Ping(…) will eventually timeout. At first, it's not super clear as to why the second ping isn't working. Unfortunately, the godoc isn't super clear on this either. It does state that all connections are cleaned up, etc, etc. But it does not state that a reconnect on the client shouldn't be attempted. I'm actually OK if it doesn't allow for a reconnect. I think it would just be good to have it clear in the godocs.

Yes, I'm aware it's open source and I can submit a PR on this. I get super self concious on these sort of things. I actually have a patch for the original mgo issue and I've been too scared to submit it. I'm not ashamed of my code by any means. I just don't want to (potentially) muck up anyone else's production code nor have I never contributed to a project outside of the companies I've worked for or my own personal pet ones (that typically never go outside of my own personal use). I then start getting into my own head and convince myself “It's just me. I don't understand it becaues I didn't RTFM properly. etc, etc…". Hell, it's taken over a decade for me to actually do a blog. I wish I had the ability to accept a PR on my anxiety issue (sorry for the terrible nerd-dad joke).

Conclusion

Well… There isn't some big witty reveal. Use it. Don't. I like it and it solves some of my issues. I personally enjoyed meeting and briefly chatting with @skriptable. He was super friendly and his talk was on the design and journey of writing the driver. He also bakes some dope ass cookies. So, if anything… go eat some damn cookies. G'night.