The clypd process

By September 8, 2013 clypd Blog, Technology
Jeff Walker

One of the challenges of setting up a development process and workflow for a startup is to fit the process to the business needs. Process for process sake is a waste of time and drives everyone crazy. The other thing that drives everyone crazy is chaos. Product hates chaos. Engineers hate chaos. I’m a big fan of “just enough process to keep us from descending into chaos”.

How much process is enough process?
We take an Agile approach to Agile here at clypd. We are constantly looking at how we do things and schedule changes to the process. Inevitably, the needle swings too far in one direction or the other. In June, we were laying the software groundwork to incorporate a new batch of hires and to crank out V1.0 of the product. This included questions like “What UI test framework shall we use?” “Do we need to test our models?”, etc. (We ended up using RSPEC and Capybara).

There was a moment where I was in a meeting with the leads and we all looked at each other and said “Crap! It’s June 5th! Is this process good enough for now? We need to crank up delivery!”. Bam. July 1st came around and we delivered our V1.0 product with more features than we originally promised.

We let the needle swing a little too far in the navel gazing direction, but we caught it. It turned out to be just the right amount of process. We delivered the V1.0 product with very few bugs.

So, what does your process look like, anyways?
Our Product guy, Jason, is embedded with the team. We all sit next to each other in the “clypd corral”. Engineering, product, literally banging elbows.Jason describes himself as a “recovering engineer”. No offense to us engineering addicts. Really. :) Because of this, he understands what it takes to dice things up into small digestible pieces and communicate the requirements/vision to the engineers.Jason is the author of all of the stories, except some of the infrastructure stories.

At the beginning of a month iteration, we have an “estimation party”. We fill up the backlog with enough stories to bring us through the month. This checkpoint forces a conversation between Product and Engineering to nail down a one month window of work/vision for the iteration.

We all sit down for an hour and go through the backlog, story by story. We discuss the “as is” and the “to be”. The “as is”, is the current state of things. The “to be” is what we expect to have when the story is delivered.

cardsThe conversation can get quite entertaining. Sometimes there is complete disagreement as to what the “as is” state currently is. Architectural discussions/disagreements often take place during these discussions. Once we are all on the same page-ish, we throw down a (custom clypd branded!) playing card with 1, 2, 4 or 8 story points.If the story is more than 8 points, we throw down the whole hand and yell “break it up! (into smaller stories)”. A story point average is taken and entered into Jira.

That’s the first checkpoint. Engineering needs enough requirements to be able to estimate the story. The estimate includes any tests, migrations, infrastructure, architectural debates, etc., everything needed to get the story delivered to staging.

Every Monday morning, first thing, we have a KanBan planning meeting where we go through the “Working Board” (current stories in flight) and the top of the backlog. We re-prioritize as needed and make work assignments. This second checkpoint forces the alignment of priorities between engineering and product on a weekly basis. This is important. When engineering and product diverge, there is waste. Waste is bad.

When someone starts working on a story, we create a new feature branch in git for that story. The engineer will push changes to that branch as they are developing the story. The engineer will also merge out from master into their branch on a regular basis.

We have what we call sandboxes. A sandbox is a combination of a feature branch, a Jenkins job (Continuous Integration) and a (collection of) VMs. We have a GUI that we developed in-house to easily create a sandbox. You click on what size VM you want, the Chef role (service) and the feature branch that you want. The GUI will create the VM for you and a Jenkins job that points at their feature branch. When the engineer pushes to their feature branch, Jenkins automatically deploys and runs the tests on their VM development environment. It also does code coverage testing using simplecov/Rcov. If the story is a “real-time” story, it is also delivered with performance tests (Gatling).

When the engineer is ready to deliver the story, they test it on their development VM (sandbox). When they are happy with it, they move it to the next step in the Jira workflow and assign it to Product for sign off. This is the second checkpoint. Product needs to sign off on the story in the feature branch/development environment before it moves to the next step.

At that point, the engineer will issue a GitHub pull request for the story. A code review and test review are conducted at that time. Once approved, the pull request is issued.

When the pull request is issued, the code is pulled into master, a Jenkins job picks it up and deploys it to staging. All of the database migrations are tested. Automated tests and code coverage all run automatically, as well. Once the story is up on staging, the engineer needs to test again to make sure everything is working. This is the third checkpoint. It isn’t done done until it’s on staging, all of the tests pass, there is acceptable test coverage and Product is beaming with pride and happiness at the awesomeness
of this new shiny feature.

Once on staging, the story is “closed”, awaiting release. A push to production amounts to deploying what is on staging to production.

Final smoke tests are executed on production after the deploy, just to make sure that we didn’t screw anything up.

This may seem like a pretty mature process for a startup. All of the startups that I have done have relied on chaos as the process. Our process has very little overhead, as it turns out. The key was to get ahead of it early, before we started adding people. Focus first on the pieces that add the most value with the least cost. Hire against DevOps early. Really early. Constantly iterate on the process. Don’t let it go stale. Make the process fun. This is key. If the process is a drag, people will just ignore it.

Our next process improvement will be to close the loop on estimates and actuals. Watch this space!

Join the discussion One Comment

Leave a Reply