Background
A few months ago I started to dig into Rust, and one weekend I successfully re-wrote one of my Golang apps in Rust!
tl;dr:
This is a CLI app that I originally wrote in Python several years ago for a project. It’s just a small utility that will attempt to resolve all possible IPs for a given domain name, hopefully triggering round-robbin DNS. It does this by querying multiple resolvers several times (Google, Cloudflare, and your local resolver). Google and Cloudflare are queried via the DNS-over-HTTPS endpoint (sending a HTTP request and parsing a JSON response) and the local resolver is just using a lower-level socket call. These requests are made asynchronously in order to be most efficient and the resulting IPs are deduplicated and presented to the user in the output. The reason I’ve now written this utility 3 times in different languages is because I think it does a good job at exposing several language features, and things I would commonly use in a language (web requests, async processes, hashmaps etc.).
My experience learning Rust (so far)
I’m still certainly a Rust-noob, but so far I’ve really enjoyed learning the language. Features like the borrow checker are truly amazing concepts, and I think Rust forces you to write good code and think about your allocations without explicitly making you deal with them. I’ve also really enjoyed Rust enums and various types like Result and Option are really interesting ways of handling errors and null. Cargo is actually a really great bit of tooling, and it makes starting a new Rust project very approachable. The ecosystem of crates.io is far more impressive than I had previously given it credit for, and it is a very smooth experience consuming crates in your app. The compile messages are amazing - there is so much that the compiler warned me on (with very detailed fixes) that saved me a lot of time and ultimately optimized my code. I could definitely feel how the compile times are a bit slower than Golang, but it really didn’t bother me while writing this app. Now, on to specifics of the re-write and comparing it to the Golang counterpart.
HTTP requests and JSON parsing
Rust did not seem to have any sort of native HTTP client, so I used what seemed to be the most popular called reqwest. Reqwest was easy to use, and once I figured out a few nuances like renaming struct values for the correct JSON key, it is really seamless to make the request and deserialize the response into the struct I defined. It was a bit strange for me (coming from Golang) that I had to search for a crate to do both the HTTP request AND the JSON parsing (reqwest and serde) because both of these features are built in to the standard library of Golang. I think I liked the Golang experience a tiny bit better because I was able to create a whole request object and pass it to another function, where I seemed to only be able to pass a client object in reqwest, but it was a very similar experience and honestly it might be possible in reqwest (I just don’t understand how). The piece I liked a bit more in Rust was the ability to just add .json() to the request and have it deserialize into the struct. That’s awesome.
Async
Async was… a bit of a struggle for me. The goal is relatively simple - execute three functions at the same time, and deduplicate the results at the end. It seems like Rust has numerous approaches to async, and it was a bit confusing to me. I wanted to keep this as close to “vanilla” Rust as possible, but the only way it seemed like I could use reqwest async was to bring in a tokio crate. To be fair, this seems to have a ton of industry support, but I couldn’t help but wonder why I needed this to do something that is also built into the language? Perhaps there’s a way, but many hours and so many sad compile errors later, tokio seemed to be the path of least resistance. In the Golang version, I use channels to have a blocking communication back to the function that deduplicates the results. This seems to be available in Rust too, but for the life of me I could not figure out how to exactly replicate the same behavior from the Golang app. It seemed like it had something to do with tokio and the response type, but I just couldn’t figure it out (please leave a comment if you can tell me where I’m going wrong, I would love to know!). I ended up running all three functions using a crate called future (another third party crate?!) that effectively runs them all, and will await til they’re all done. Once they are all completed I then run the resulting hashmaps through the deduplicating function. Although I seemed to get this working successfully, Golang is a clear winner on this front. You literally just put the word go in front of your function call, and it’s running asynchronously. There were so many rabbit holes I went down trying to get this to work in Rust, and it seemed unnecessary that I needed two external crates to accomplish the job. That being said, I am a Rust noob, so I’m sure I did something less than optimal, and any feedback would be appreciated! I’m going to continue looking into this, and hopefully optimize it a bit better in the future.
Dependencies
Golang is built for this sort of work. The entire app in Golang has ZERO external dependencies - it just uses the standard library. Everything from HTTP (requests and server), JSON parsing, async, and socket requests are all built into the language. To be fair, Rust was certainly not designed for these same use-cases, but it was a bit eye-opening to see that I had 127 external dependencies for my simple app.
Docker
I expected this to be easy - my app was compiling locally just fine, and I had finally removed all compilation warnings. I have written literally hundreds of Dockerfiles in my career, and it looked like there was even an official Rust image! Well, this ended up being a several hour long rabbit hole, and it’s all because of OpenSSL. It turns out that many libraries (in my case specifically: reqwest) needed to be compiled with OpenSSL which is apparently terrible to do in an Alpine image. After multiple attempts to apk add the compilation dependencies, I finally got it to compile in Alpine! When I tried to run it however, there was just a very obtuse error about a core dump, and really nowhere to troubleshoot from there. I took a few other approaches including a guide that basically said “compiling Rust stuff in Alpine is terrible, so use a Debian image and cross-compile it for Alpine!”. I never got this to work either. I finally took a peek at an approach I’ve seen in another Rust project using ekidd/rust-musl-builder image which did actually work perfectly. However, this is NOT the official Rust image, and looking at the project a bit closer, people should probably stop using this image (it hasn’t been updated in 7+ months, and there is an open issue asking if it is still being maintained: https://github.com/emk/rust-musl-builder/issues/147). I finally gave up on getting an Alpine-based image, and switched to Debian slim. It took a few attempts, but finally after installing the right compilation dependencies (and CA certificates for the final image) I finally had a working image! Debian slim is a bit larger than Alpine, but it feels better security-wise that it’s built from an official Rust base image. This is a HUGE contrast to the Golang docker experience. I basically just copy/paste between projects because it’s almost exactly the same every time. I guess Golang has their own TLS libraries in the language itself, so OpenSSL is not required for anything I’ve come across in order to compile.
Final Thoughts
After everything, the Golang version is still faster than the Rust version (the Rust version is still way faster than the Python version though). I’m guessing it’s because I’m not doing the async pieces as optimally as possible, so I will probably try to figure that out at some point. All in, this was a fun weekend hack and Rust is an awesome language - I’m glad I’m learning it. I will likely continue building with it and hope to be less of a noob some day. That being said, I would HIGHLY encourage those of you who have played with Rust to give Golang a try as well. Golang is much easier for some of these common system functions, and while I know Rust can be more performant than Golang, it’s certainly not for free. I hope this was interesting to you all!