Breaking Up With the Monolith


Screen Shot 2015-09-18 at 3.23.28 PM

(An ancient Persian monolith bearing a striking resemblance… -- Creative Commons Licensed Image)

For years now, Craftsy has been innovating on an ever-growing Java monolith that is our web platform. During that time we cemented together our online classes platform, e-commerce, patterns, projects and more. It worked and served us well through the years!

But, over time our integration testing grew to requiring days. Releasing changes to the monolith took hours and involved multiple stakeholders all in a room simultaneously to ensure deployment success. Hotfixes between releases were frequent. Our awesome QA team worked their fingers to the bone, sometimes with late nights and weekends, so we could get releases and fixes out the door for our customers.

We have an amazing customer base of creative makers and we have to adapt quickly to meet their needs. It was obvious from our limitations with the monolith that we had to do something so we could innovate faster. Our teams needed greater flexibility to do A/B testing. We needed to update and integrate the user experience between our classes, e-commerce, patterns, blog, and projects to fit our customer’s needs. We needed to make the user experience more inspiring for our creative customers.

Enter Craftsy Web 2.0! Opportunities to do a rewrite of core products are a rare privilege. The new Craftsy Web architecture focuses on Domain Driven Development and will allow our Product team to work tightly with Front End in Javascript/Node while the Platform team focuses on the API Aggregation and domain services on the backend. Swagger contracts will assist in speeding up development. Unit, load and smoke tests are required as part of development to facilitate safer and rapid releases asynchronously from the other moving parts.

In this new environment we’re deploying immutable AMIs to AWS from Jenkins, using Packer (and Vagrant for developers) with Terraform plus Consul for service discovery to facilitate easy deployment and horizontal scaling. The F5 load balancer and web application firewall fronts our new environment and HAProxy (or the F5) balances traffic to the API Aggregation layer as well as the backend services.


(Draft Architectural Diagram for Craftsy Web 2.0)

The move from one giant Postgres DB to service specific DBs is huge. The data migration strategy from our monolith to this new architecture to facilitate deployment in multiple geographic regions is a massive project by itself! To start this off, we will be generating business process documentation and mapping data flows as we work on data models.

We’re using configuration management (Puppet) alongside Packer to provision files that will be baked into immutable AMIs. We may also use Puppet occasionally (single run - not daemonized) on the DB instances to facilitate maintenance and upgrades. Although we’ve talked about using other config management solutions, we’re using Puppet on a number of other long-lived (non-scaling) cloud and local infrastructure systems. After internal discussion, we’re tentatively going to continue to use Puppet.

There is a lot of neat stuff to get excited about, and we’ll be writing future blog posts about our decisions and process. This really is a great opportunity driven by our business needs, and we’re just getting our feet wet. Let us know in the comments what topics and areas you would like to hear more about. We look forward to sharing more with you in the months to come about Craftsy 2.0!

Comments (0)

The comments to this entry are closed.