The Chef Win Series is a response to: "How has Chef changed our operations for the better, in the past 2 years since our shop adopted it?"
Some additional perspectives:

In this article I will outline what many of us have observed as an existential threat to Chef: Dockerfile, and what I would propose as a remedy.

The worst part of Docker is also a natural fit for Chef. That's an important opportunity, in what could otherwise be a dangerous situation for Chef.

New Chef Challenge: Immutable Containers

For the devops novice: a fair warning - this is going to get into a black belt area - and I am no black belt in this area. Consider the source before taking this information too seriously.

First things to learn about devops

  1. Manually built servers - bad
  2. Automated servers - good
  3. Golden images (server image) - bad 

That "Golden Images" thing was the kicker, because a golden image implies lack of automation, manual tweaking, configuration, etc. But then this Docker/container thing comes along, and now there is an opposing concept:

  • Immutable preconfigured instantaneous container images - good (???)
  • Automagically configured (Chef/Puppet/etc) VMs - good

WTF? This is too big a topic for this blog, but now there is a growing perspective within the DevOps commnity where you really don't want to use Chef to build your servers, but instead use a container image (and do your configuration in Dockerfile, etc) - but only when your server is a part of a VM, not the whole VM. Confused yet? Good, you should be.

  • If container image - configure as immutable with Dockerfile
  • If complete VM - build with Chef/Puppet/etc

If you don't see here a dangerous threat to the entire Chef market, you're just not paying attention. More so because the second category - apps such as databases, which consume the entire VM - are themselves the next focus of containerization, and may eventually disappear to zero.

The scary part for me is that many of the people I respect most regard Dockerfile as an existential threat to Chef. This is not good news, to me, on several different levels.

Got That? Good. Next?

If your head still isn't spinning, then you're faster than me on the uptake. But when it quits spinning, I'm still not finished arguing. The problems are these.

  • Base images are the weak link in container land (Docker, CoreOS etc)
  • Building containers from base images with Dockerfile is an odd place to start

Docker moves very quickly as a project, so this information may be wrong or irrelevant soon. But my experience is that consuming Docker is entirely dependent upon images which may not exist, and worst, depend on images that you have absolutely no reason to feel confident about - either security-wise, or other. Assuming that the lack of confidence in existing base images stays unchanged, you need to be able to build your own base images, and this is not - or at least at the time of this blog - a trivial undertaking.

This topic is enough for an entire book - so I'm not doing it justice here. But if you've worked with containers in a serious way, you're already familiar with the compromises generally accepted in the current state of the art in building and consuming base container images. Enterprises cannot be expected to simply accept these compromises. They won't.

My opinion only: Chef should be a primary source for container base images. Perhaps this has happened already but if so they've done a good job of keeping it a secret. 

This capability to create container images should be THE most bulletproof, most tested, most perfect, most publicized capability Chef has. To do otherwise is to cede the entire market to Dockerfile, and leave the excellent capabilities of Chef totally in the dust. My 2c.

In my crazy world view, Docker is a great tool. But building images should be done as much as possible outside Docker, for the same reason that bash scripts were not the preferrable approach to building servers before Chef/Puppet/Ansible/Salt.

There are many people who are much smarter than me working in this area. But from my perspective, this is a slam dunk conclusion.

Chef's Container Story a Complete Disaster - So Far

One of the coolest features of Chef the company is their ability to completely misfire and then act like nothing ever happened. Tone deaf. Drives me nuts, but it is sometimes helpful. To wit:

Knife Container was a complete misfire and is now deprecated. But only recently explained as such. How confusing is that?

Chef still does not appear to "get" containers at the corporate level, nor are they marshalling resources to meet this existential threat. [read: Embrace and Extend - the old Microsoft strategy]

The Chef for Containers page has only recently been updated (and apparently in response to the original version of this specific blog WTF ?%&$*#!!!) and begins the briefest recap of Chef's current container story - but it's much better.
docs.chef.io/containers.html - Check it out for yourself.

Containers seem to be moving MUCH faster than Chef's work with same. This appears to be pure and willful hubris - to me anyway. Only Docker's greater immaturity and hubris would prevent significant damage to Chef's market position as containers grow in popularity - or at least if Chef doesn't respond in a less tone deaf manner. My 2c.

What Will The Market Want?

Chef is counting on the fact that market forces always move towards a single tool, and Chef is still a better choice in the enterprise than Docker to manage an entire enterprise. Unless of course it doesn't even work for containers. Woops.

For shops like ours, this is more than just a convenience. If we can't do it best with Chef then our entire story to our already confused customers becomes disjointed. Makes us look like idiots. "Oh? You are still using Chef? How very 2014 of you!" Last year's bleeding edge is today's bandaid - crusty with dried blood, and cast in the waste-basket.

The market will prefer Chef managing containers, but only if Docker (and Dockerfile) don't beat Chef at it's own game. At one point or another it's just easier to leave Chef behind. The 2015 containers lecture circuit already has. That's crazy.

Summary:

  • Docker - good.
  • Building docker images from other docker images? - questionable.
  • Chef's container story? Emerging - but immature and way less serious than it should be - given what many regard as the existential threat of Dockerfile.