SurgeCon Review – our thoughts on a fabulous web scalability conference
Last week 3 members of the SEOmoz dev team (Walt, David and I) traveled to Baltimore to attend SurgeCon. This conference was all about web scalability – including architecture, operations, and...
View ArticleTerror-bytes! Or, ‘How we use Amazon Web Services at the terabyte scale’
Hi! I’m Chas Williams, one of the developers working on Linkscape at SEOmoz. Here at ‘the moz’, developers are encouraged to participate in the tech community at large (by attending conferences), as...
View ArticleHigh Performance Libcurl Tips
At SEOmoz, we’ve been able to make use of many great open source packages. One that’s particularly important to our crawling infrastructure is libcurl, which abstracts the logic behind HTTP (and many...
View ArticleHow to cache HTTP range requests
Executive summary We have a highly parallelized service that makes thousands of simultaneous requests to the same S3 object. When we request the same object too many times S3 starts to throttle our...
View ArticleDeterminism and the Semantics of Performance in the Cloud
SaaS companies are judged harshly by the perceived reliability and performance of their services. When your service becomes a critical part of a customer’s infrastructure, their fate becomes wedded to...
View ArticleSSD drive failures
As you may have read, our Rankings service has recently been afflicted with an SSD-related technical malady. Specifically, we were using SSD drives in a heavily read/write environment to be super-fast...
View ArticleGod & Chef vs. Screen: How Fresh Web Explorer runs workers
Over the course of developing and maintaining the crawling and processing pipeline Fresh Web Explorer, we on the dev team have encountered a tradeoff between between two goals, each of which is more...
View ArticleTCO: The Nerd’s LOB
Welcome to the first in a series of blog posts we’re planning around Moz’s adventures in cloud computing. You may have heard of the Seahawks’ “Legion of Boom” (LOB). It’s the nickname that the...
View ArticleGod & Chef vs. Screen: How Fresh Web Explorer runs workers
Over the course of developing and maintaining the crawling and processing pipeline Fresh Web Explorer, we on the dev team have encountered a tradeoff between between two goals, each of which is more...
View ArticleTCO: The Nerd’s LOB
Welcome to the first in a series of blog posts we’re planning around Moz’s adventures in cloud computing. You may have heard of the Seahawks’ “Legion of Boom” (LOB). It’s the nickname that the...
View Article
