Hey all, I am prepping currently a talk on Microservices. Here’s the references I used (very heavily) as sources. I owe these thought leaders a great debt, as they’ve experienced many of the mistakes and pitfalls that can doom a migration to this challenging but enticing model. You must, MUST read at least the first three sources as they’re definitive. And maybe finish up with Jez Humble’s seminal “Continuous Development”?

Thanks as well to Joe Dunn from Marquam for inviting me to be a speaker. Joe, I owe you one buddy, what an honor. Thank you!!!

The Big Three

1. First is the best IMHO and it sets the tone for everything that follows. Martin Fowler, https://martinfowler.com/articles/microservices.html – essential reading. Includes prerequisites, stumbling blocks, positive aspects of the design. I love Martin, he’s real world but can see the big picture and has lots of experience. The one page omnibus is here: https://martinfowler.com/microservices/
2. For governance and a very pragmatic balance between safety and velocity with your architectural reviews, “Production Ready Microservices” by Fowler. Microservice evaluation is organized by topic. This is real world and based on her experiences at Uber. Thoughts on alerts and dashboarding are solid and well thought out. You can crib from this yourself in building a checklist for your audits.
3. How micro is micro? A great discussion on Domain Driven Design, chapter 5 of “Microservice Architecture Aligning Principles, Practice, and Culture” by Nadareishvili et al – Microservice Architecture. It includes a great practical breakdown of handling one workstream and defining service boundaries using DDD of a sample company.

Secondary Sources

1. How Soundcloud migrated from their monolith using the strangler fig pattern – three parts, first one is the best IMHO – Phil Calcado: https://developers.soundcloud.com/blog/building-products-at-soundcloud-part-1-dealing-with-the-monolith (part 2 is here, part 3 is here)
2. How complex architectures like Netflix are in reality – https://www.slideshare.net/brucewong3/the-case-for-chaos
3. For an end to end Microsoft implementation Bob Familiar’s book on “Microservices, IOT and Azure” is very solid. I wish I would have had more time to include his walkthroughs on containerization with Docker.
4. Building Microservices by Sam Newman. Again, time didn’t allow me to use his material much, but it is VERY valuable here, and very explicit/tactical in its approach. I love it!
5. Reactive Microservices Architecture – free ebook, design principles, very Java based. This seems to tie in very well with M Fowler’s thoughts on designing for failure.
6. Microservices with Docker on Microsoft Azure, by Scholl / Swanson / Fernandez.
7. Video from Chicago in 2015 – “How I Learned to Stop Worrying and Love Conway’s Law“. James Lewis. I love this one because there’s a few examples where they knew the org was not capable of the change needed – and designed a system that would fit it (square peg in square hole!) instead of dictating how the design should work in a perfect, idealistic world.
8. Zhamak Dehghani – Real World Microservices video – holy smokes this is old, all the way back to 2014!
9. Randy Shoup on Microservices and Conways Law (video and transcript)
10. Implementing Domain Driven Design, by Vaughn Vernon. Decomposition and finding domain boundaries.
11. Video by Sam Newman, outstanding. https://www.youtube.com/watch?v=PFQnNFe27kU
12. A free ebook on this page as a PDF – excellent resource for Docker containers and .NET architecture. The ebook is here. The same page has a ref to a “Containerized Docker Application Lifecycle with Microsoft Platform and Tools’ book that looks excellent. https://aka.ms/DockerLifeCycleeBook
13. Yes there are ways to make transactions work in a distributed system. It just takes some work, but there are some usable patterns out there. (I think it’s hard to argue that it will be harder to debug and implement than with a monolith however!) Here’s one article on the subject, with a nice breakdown of DDD to boot.

 

The Cheat Sheet – Who, What, When, Why.

What are Microservices?

There’s a few definitions out there, which seem to complement each other.

Short version –

• “A software building block that does one thing and does it well.”
• “Microservices are small, autonomous services that work together.” – Sam Newman, Thoughtworks Adrian Cockcroft, “Loosely coupled service-oriented architecture with bounded contexts.”
• Ideal for big systems (new problems arise because of their scale – beyond boundaries set by initial architects.”

The upshot is that this is a design that values replaceability over maintainability – i.e. cattle not pets. Architecturally it borrows heavily from SAAS, Continuous Delivery, Agile, and DevOps. And yes even SOA!

What are the benefits of moving to microservices?

There’s actually a lot of benefits with this type of architecture – and unlike SOA there’s a few real world success stories we can point to (Netflix, Amazon, and Microsoft etc) that all decreased release sizes and improved maintainability using a microservice architecture. It allows technical diversity (I can write my code any way I want using different libraries as long as the API layer is consistent), independent deployments (a huge win, suddenly I am unshackled and able to get my bits out the door faster without waiting on tightly coupled partners and huge testing cycles), open standards, and flexibility.

Uh huh, I’ve heard that before. What’s the catch?

Microservices aren’t a silver bullet and anyone telling you it is is trying to sell you something! They are much, much harder and more expensive to monitor and control. They seem to be a better fit for large complex systems, at companies that stream content and are otherwise loaded up with talented staff with a high level of proficiency both in the business domain and in DDD theory. It’s hard to prototype Microservices too – one quote that sticks out in my mind is, “the decision to move to microservices must be company wide.” Agile has been very successful as it is internal to one team; DevOps less so as it requires cross functional groups. Microservices, with an incredibly complex and hard to troubleshoot operating environment that is constantly in flux and rarely consistent (i.e. a debugging nightmare), will often make your siloes of knowledge worse not better and if you’re not prepared to enforce governance it can make maintenance worse not better and create a spiraling technical debt issue as teams work in isolation without considering quality or consistency – the exact opposite result you’d expect with this type of architecture. Again, see Susan Fowler’s terrific book on “Production Ready Microservices” for a roadmap on enforcing consistency for more on this.

I love it. What do I need to do to get here?

Well, you’re going to need to do a health check on your readiness before you make the leap to microservices. How long does it take you to provision an environment? Do you have a templated set of builds that are in source control – i.e. infrastructure as code? How capable is your monitoring? (You’re not going to have a lot of time to track down issues, alerts must be actionable and real-world and your devs onboard with handling support 24×7). How automated is your release process, and can you run deployments out to production – yes gated at QA – with a high level of certainty on quality? (This implies a very well thought out and automated test strategy. You can and should think feature flags, canary testing, etc – but the tighter the cycle from dev to prod and the more automation, the better your ecosystem can handle the leap to MS.) Do you have a top down commitment to quality – are your executives/business stakeholders comfortable with partnering with you and viewing quality as a worthy investment? Last – how far along are you on your DevOps journey? Can you form cross functional teams or is there a wide gulf between you and your siloed IT/Ops partners?

Eeeeshhhh. We’re just not there yet.

Good! That’s a sane reaction and it’s a very good thing to look before you leap. Microservices are awesome but still very new and we’ve already seen enough to know that they will not work in just any organization. (thank you Conways’ Law!)

That being said, there is much you can do to improve things without making this kind of a wholesale leap. For more, check out this post by Donovan Brown. In essence, he says to start with your pain point. Maybe your agile SDLC is humming right along but your operations team is struggling to keep up. Or maybe it’s taking 3 months to provision dev or test environments, and they don’t match production. Or perhaps your RM process is manual and error prone.

The roadmap that Donovan discusses starts with the following:

• Source Control
• SDLC – Kanban boards, accountability to business, unit testing, team definition of done.
• Automated deployment to dev
• Automated testing – automated unit tests gating out to QA, not PROD we’re not crazy, and a sane level of integration testing
• Infrastructure as code – no more hand coded or tweaked environments, we’re using templates checked into source control to enforce consistency. We’re not patching, we’re replacing!
• Monitoring – no more drowning in a barrage of alerts, but actionable alerts that give us a heads up before availability issues arise
• …. And rinse and repeat. Tie this into your business values and try to show KPI’s so you know if your DevOps improvements are gaining traction or if you are just spinning your wheels.

That’s all folks….

I will continue to update this. And I’d value your feedback!