Sonalake @ ShipItCon 2019

Friday a week ago 5 of us Sonalakers attended ShipItCon 2019. This nonprofit annual conference in Dublin’s Mansion House focusing on software delivery provided a break from the norm for this hard-working group. The action-packed one-day event featured eleven speakers, all with their own highly detailed well thought out story to tell. By the end of the day, the Sonalake crew had a lot to compute. That evening this thirsty bunch retired to the smallest pub in Ireland to debate our three main takeaways from the day. Here’s what we eventually agreed on.

Observability is super important

The keynote speaker, Charity Majors from honeycomb.io set the scene for the day with the opening talk highlighting the powers of observability. Charity’s key point is that as applications become more complex, simply using metrics to detect or prevent errors will no longer be sufficient. Metrics are unlikely to help deal with the kind of catastrophic outages that only happen once. When we have a failure we need the ability to examine what occurred prior to it happening so that we can identify the true root cause as opposed to simply implementing a workaround. What we really need is the ability to debug the errors after they have happened. After any failure, a developer should ask themselves is there enough information to figure out what happened?

Laura Nolan, Senior Staff Production Engineer at Slack, expanded on Charity’s point by first talking us through some catastrophic failures experienced by large organisations and diving into what caused them. In each of these cases, it was observability that allowed the root causes to be identified.

There are common practices that contribute towards effective observability, and many powerful tools available to help externalise key application processes through logs, metrics and events. Using these goes some way in improving observability, but to achieve an optimum level, a real culture shift is required. Developers must factor in observability from the beginning and throughout development ask, how can I make this system more observable.

Observability is super important!

Immutable deployments are cool

Aaron Chambers, Senior Engineer at Phorest, gave an overview of their deployment processes. Their static frontend code that is uploaded to an Amazon S3 bucket and the index.html is then cached Redis keyed by its git SHA. One key is then marked as “active” and this is the file that production site visitors users are routed to. Interestingly, other versions of the index.html file can be accessed by passing the git SHA as a URL parameter allowing users to access different versions of the application.

With this mechanism, any build can be pushed to production, as nobody will see the version unless a user specifically asks for it. This approach allows for new features to be “canaried” slowly into the real world. Phorest adopted this approach to enable them to test pull requests in production, and allow some production users to verify pull requests before they’re even merged. How great is that!

On a related note in a follow-on talk by Mihai Criveti on continuous deployment, he pointed out that immutable deployments are incredibly convenient they aren't entirely safe as data isn't immutable.

Monolith | Microservices | Serverless

Microservice and serverless architecture adoption have grown in recent years, with many organisations realising significant scaling advantages. Others remain loyal to a monolithic architecture. ShipItCon 2019 by no means shied away from this debate featuring speakers from all camps.

Sam Newman provided an informative view of microservices and serverless architecture, explaining how to split responsibility across different elements. For example, a microservice is a logical entity that owns its data, but it may be built up from multiple serverless functions that share this data.

Sam defines serverless as being a logical abstraction with multiple layers including serverless, service, and hardware abstractions. Ultimately Sam is made the point that with these layers of abstraction that you are only able to define the simple things like what JVM or Node version you are using, or how much memory you want. There’s less for a developer or ops person to deal with, but there is also much less flexibility and more vendor lock-in the lower abstract layers.

Ben Lavender, a software engineer at Stripe, presented the case for choosing a monolithic architecture. Ben provided an insight into how Stripe manages their codebase - one large Ruby repository that results in a single gem, with hundreds of developers work on it concurrently. Ben went on to detail two rules he uses for deciding when to split out a “macro service”:

• Prefer code interface at logical boundaries
• Prefer network interface at operational boundaries

ShipItCon 2019 most definitely provided an energising and educational day - all 5 of us took something from it. It not only covered some of the highly opinionated topics of software delivery but also provided very informative tips. Two standout ones were a neat technique for identifying what technical debt to tackle first using a tool called code-maat, an introduction to CNAB, and a thought-provoking presentation on diversity. The lunchtime stroganoff wasn’t bad either!

Kudos to the organising team for the great event and for donating all profits to charity. Well done and hope to see you all at the next ShipItCon!