Stop thinking Agile, and start thinking agile

There's been plenty of sentiment about Agile (capital A as my buddy Ed pointed out to me) being dead or that it's a failure or that it's in some state of decay.  In a sense, I completely understand why people would think that.  It may also be some of what is holding back the free thinking in the process of software development.  There are few big, new ideas (outside of lean but then that's not new) that are really providing direction for the software development process.

From my point of view, this wound is self inflicted.  Agile has become the hammer to nail all process problems.  It was championed and supported by some really smart guys in the software dev space.  Popularity grew, Agile become all the rage and, as all fads in the space, it became a personal point of pride to declare that you or your company "went Agile".  There were books written about it that contained all the activities and routines you must partake in in order to be considered Agile.  Training classes and certifications inevitably popped up that claimed that with 3 days of training you too could be a scrum master and Agile wizard.  Before anyone knew it, Agile was a defined creature with a strict set of rules (standup meetings in the AM, planning poker, TDD, pair programming, retrospectives, etc.).  It was no longer the organic process that it was supposed to be.

I certainly wasn't there in the early days of Agile but my company did "go Agile" almost 3 years ago.  With it, I've found some fantastic practices that are now part of my software development six-demon bag and I've chronicled many of them on this blog.  It's truly been a great addition to my craft and I would certainly advocate others to try it as well.  GO AGILE!  You may thank me later.

So what's the problem here?  I just said Agile prevents free thinking but then I advocate that you try it.  The problem isn't the Agile movement, it's how it is adopted.  People want to add it to their process without appreciating where it came from and why your company should care to use it.  It's the same behavior I allude to in my "The Goal" entry.  Blind adoption of the Agile process isn't going to be to your benefit.

When I talk to people about a good software development practice, I no longer think in terms of the canonical Agile.  Every company, every product, every team have their own unique environment and set of problems.  It would be a mistake to not take those things into account before you start throwing the Agile bible at everyone.   My goal is to streamline the process and use the best tools I can to reach the coveted hyper productivity.   Agile is comprised of many different practices and procedures.  Take what you need from it and recognize what works best for you.

Remember, Agile wasn't created from thin air.  It was coined by a bunch of guys that saw waste, friction and deficiencies in the process.  This is not uncommon to any of us.  These are things we all see in our days, isn't it? "Wow, that meeting was useless."  "The number of regressions grows on each release!"  "We build what they ask for but then they never like it!"  "These manual deployments take hours and they're error prone!"

LISTEN to those thoughts and react to them.  As a developer I know how easy it is to identify what is going wrong and grouse about it.  It feels great to vent.  If I asked you what pissed you off, you'd ramble about it for hours, eventually repeating yourself but then you'd dive back into it anyways because you're that passionate about it.  Take that energy and think of how you would fix things.  If that meeting was useless, how would you make it better?  Should you even have been there?  Were there too many people talking at once?  Was there one loudmouth that dominated the whole discussion?

Don't accept this reality.  Change it.  Make it better.  Be organic in your process.  Evolve.  Mature.  You don't need a certification to understand where things are going wrong for you.  Fix it.  You have manual builds today.  Automate them tomorrow.  You have 6 month projects with unhappy customers.  Decompose them into 1 month deliverables to get quicker feedback.  You have too many bugs.  Increase test coverage and improve your QA.  You have a schizophrenic product organization that pulls you in 8 directions?  Stop the assembly line and demand consensus and priority.

That's all Agile ever was.

...but before you go, buy a book, ask questions on forums and learn about all the wonderful practices that fall under the Agile umbrella because we've learned a lot in the last decade about good processes.

The Goal

The only right way to write code is to use TDD!

Scrum should only take 5 minutes!!

You should ALWAYS use DDD to develop your model!!!

NOSQL is the only way!  Down with SQL!


That's the sentiment you can find in a lot of arguments between developers.  It's too black and white, too absolute.  The issue is that the focus leaves the goal.  It artificially limits the solutions you can come up with.

In the case of TDD, you can most certainly write code before tests.  What do all software development practices boil down to?  Building easily maintained software.  It isn't about creating beautiful code, it isn't about designing a family of classes, it isn't about creating a model that sounds like a spoken language.  Building easily maintained software can be done in any number of ways.  Something like TDD helps to reinforce proven practices for maintainable software but it's not the only solution.  I'm a TDD practitioner but you know what?  I've got no beef with writing code first.  You don't always need to TDD.

In the case of agile development, for anyone adopting agile, there's a LOT of focus on the religious practices that come with it.  You will, for a long time, follow these practices blindly, even when they don't improve your process and become a nuisance.  The answer is that not all practices may make sense.  Take a step back and look at what they are intending to achieve.  A 5 minute scrum doesn't mean you're doing a good scrum.  You may find that, for your group, have something like 8-10 minutes is more comfortable.  Don't hate yourself because you're not following it to a T.  You've done exactly what these practices were intended to do; maximize your productivity.  That's the goal of this whole thing.

In the case of NOSQL, it's a poor musician to blame his instrument.  People finally realized that they were using SQL as the hammer to nail everyone's persistence problems.  With the host of new applications we build, SQL isn't always the answer.  Maybe it's a document database.  Maybe it's a key-value store.  Maybe you need something that natively supports object graphs.  Instead of recognizing the fact that applications and their persistence mechanisms come in all shapes and sizes, the sentiment has become that SQL is the scourge of software development.  We have ourselves to blame on that one, trying to force an object model into a system that poorly represented it.  Somehow this is SQLs fault?  We stopped questioning why we did the things we did.  Take a step back, ask yourself what you're really trying to do and then it becomes a no-brainer.  That's what gave birth to all these new forms of persistence.

For some, it may take a while to see it but you'll get there.  Don't fall into the trap of blindly doing things without re-evaluating constantly.  There's more than one way to do something and you shouldn't let software purity or zealotry prevent you from considering it.  It leads to the innovations that improve our lives as developers and the people who use our software.

It was a busy summer

This last summer I spent time getting embedded in a new group at work.  It's been a blast because the group is really enthusiastic about agile development and adopting new and efficient practices.  I've had a chance to practice a lot of the things I've been doing and studying in my free time including acceptance test driven development, NHibernate, ASP.NET MVC 2 and lots more BDD.

There's nothing sweeter than having a greenfield project and I was able to get my hands on one.  I was tasked with building an internal dashboard for some of our back end (Windows) services.  I'm still in .NET land and we're primarily a Microsoft shop so I used a pretty small but flexible stack,

• ASP.NET MVC 2
• Fluent NHibernate
• StructureMap
• jQuery

For testing, I tried out doing acceptance test driven development with SpecFlow which is the .NET analog to Ruby's Cucumber.  My unit/integration testing framework of choice is still MSpec.

I took this experience and made a presentation to the company about BDD.  For anyone that's looked around the web at BDD presentations you won't be incredibly surprised by the content.  Lots of the same points are made and I even borrowed the excellent high gear/low gear analogy of ATDD I found in someone else's presentation (I can't remember who :-(...sorry, uncredited person, whomever you are).  You can download it from my dropbox here.

I know I promised an example of doing ATDD and I'd really like to do it.  Now that work is semi-calm I can try to whip something up.  There are tons of other things I want to talk about like the Java codebase I'm working in now, when NOT to unit test, the current .NET craze around CQRS, and other interesting topics swimming around in my head.  

Analysis and Investigation on Outside-In Development

BDD has a variety of testing tools surrounding it in a number of languages. Since I'm primarily in the .NET world (looking to expand that soon), I've dabbled with MSpec and had a lot of fun/success in creating descriptive tests that spoke the language of the business and communicated value. That was my first experiment in the world of domain drive design in an agile process.

The one thing these tools have in common is that they promote outside-in development. In BDD that literally means starting at the touch point with the consumer/customer/stakeholder. For most applications, that's a GUI of some sort but for others it may be a web service or something similar.

I've seen outside-in development go by a couple names like story driven development or acceptance test driven development. Unless I'm mistaken, they're both trying to achieve the same goal. What outside-in development is trying to do is make the software customer-oriented not unlike the agile processes that fit with this thinking. You start building the software from the customer's point of view based on their specification of it (usually as a story). Coupled with agile, you're doing something along the lines of rapid prototyping (with a bit more discipline). It's also keeping you focused on what the customer wants. It discourages gold plating and extra work that are typically waste.

<tangent>

I don't like to say gold plating because I know there are those developers that think that discouraging gold plating is like discouraging free thought and creativity. I would disagree. The hard pill for us to swallow is that what we perceive to be a "better way to do things" is not from the perspective of the actual consumers of the application. They may have no use for the bells and whistles that we believe to be productivity boosters. What it may be doing instead is creating excess inventory that costs money but has no ROI.

However, if you're in an agile process (or even if you're not), don't despair! Agile, by its very nature, relies on high-bandwidth communication. Reach out to the customer and communicate your creative thoughts and ideas. They will rely on your expertise in this arena and most times accept your recommendations. But, just remember, if they don't find use for your widget/tool/interface then don't do it (at least on company time :-P)!

</tangent>

Sooooo this is what I've understood so far. I am by no means an expert on outside-in development and I'm still trying to understand the practice. In the past, I've worked from the application logic and worked my way downwards. I haven't done it from the UI down which is certainly an interesting experiment. I'm also coming to find that I haven't done nearly enough acceptance/functional testing in my travels.

I plan on following up this post with my first attempt at doing outside-in development. I need to find a good book on it. I'm pretty sure the RSpec Book is probably what I'm looking for but any recommendations are appreciated. In the meantime, I've just been reading blogs, articles and comments. One such blog I ran into has a tremendous amount of insight into testing and offers good information to get you started on writing acceptance tests using ASP.NET MVC and some other tools and frameworks. Check out Steve Sanderson's blog to get some really great information.

Too many interests, too little time

I have too many goddam things I want to check out in the world of software development and not nearly enough time to entertain them in any significant fashion.  There are a few tricklets missing or otherwise unpolished from my programming bag of demons that would prove to be valuable to my ability to dispense truth and beauty to the world of code.  Here's the latest group of stuff that I want to dig into:

• Ruby - Believe it or not I'm beginning to feel constrained in statically typed languages (it's a mostly love and partially hate relationship).  Ruby seems to be all the rage so I want to poke around in it.
• AOP - I would love nothing more than to break logging out of the components I write.  I know I can abstract them out in one of a billion ways but I'd rather just have it be completely invisible to classes whose responsibilities should be focused on their task and not making sure log4net gets the message.
• Being a computer scientist - I got so wrapped into the abstraction of software that I stopped thinking about the science of it.  Steve Yegge reminded me just how much rust I've accumulated since graduating.
• Agile Development - I've read the manifestos and some online documentation but I haven't committed to reading more details or purchasing any publications.  The only reason I'm partially reluctant to start now is that it's a major uphill battle with my current team.  They aren't dumb and they read about this stuff but there's no interest there.  I'm not much on lobbying and I can't train them all (including the role of our manager).

Right now I'm mixed between finding out what AOP offerings there are for .NET and playing in Ruby development.  I want to play with toys instead of read books.