For all the ranting and raving I do about agile and how it's the best thing since sliced bread, you might think that I am of the belief that in order to create great software you HAVE to be doing agile.  After re-reading my post yesterday about the sloppiness of Agile, I realized that the post, and likely many others I've written, could lead people to think that I believe the only way to create great software is by pair programming, having weekly iterations, having retrospectives, and doing everything else that the agile guide book tells you.

Not so.  While I do believe that the agile principles and practices that many of us follow greatly increase our chances of success, they are by no means a silver bullet, and they are by no means a prerequisite for great software.  If you're doing waterfall, RUP, scrummerfall, or whatever else and creating software that your customers love, then keep rockin the process you're using.

My experiences give me a bias towards agile, and the many practices that it recommends.  I've seen it work well in ways I don't think waterfall would have.  I also really believe in many of the core coding practices that agile recommends.  Most especially: test driven development (TDD), continuous design, refactoring, YAGNI, and keeping it simple.

I think agile gives me a better chance at creating great software and that's why I use it.

warning: this post contains higher then normal sarcasm levels

Over the last couple weeks and months I've had a number of people share what seems to be a common Agile misconception.  I've never heard anyone say this first hand but my impression is that it goes something like this.

Agile lets programmers do whatever they want, they never have to ship, they don't have to follow anyone's rules, and they can be as sloppy as they want.  Cowboy coders of the world unite....oh and sign this here agile manifesto while you're at it! 

Having worked in an agile shop for the last 3 years, as well as advocating and making small inroads at my previous employer for a couple years prior, I feel like I'm reasonably qualified to speak to whether that statement is true or not.

To be honest with you....it's completely true!  I am proud to say that over the last 3 years we've created some of the crappiest code known to mankind.  We haven't shipped anything to anyone since we get to decide when the software is done, and well, we've had much better things to be doing.  We make all our own rules, which we've found best to make up on the fly.  It's really a great setup.

There is just one thing.  For one, we do this thing called pair programming.  If you haven't tried it, it's where two programmers sit at a single computer and write code together.  We do it close to 85% of the day, and the vast majority of our production code has been written in this fashion.

There is something strange about pair programming.  You see when you program alone it's very easy to be tempted.  Tempted by the internet, by your email, by your shinny new iPhone.  It can also be tempting to take shortcuts in your code, do things the "easy" way instead of the "simple" way.  As long as it works and nobody else has to look at the code again, who could it hurt if it's mostly unreadable?  As long as that bar goes green we're good, right?  Actually, if we get rid of the green bar, we don't have to worry about wasting our time making it turn back green when it suddenly decides it wants to be red.

Pair programming doesn't allow you to be sloppy.  Actually that's a lie, it does, but it makes it twice as hard.  Not only do you have to be a slack ass, but your pair also has to be a slack ass.  And you and your pair both need to not care about the fact that you both know you're being slack asses.  And you and your pair need to also not care that when the next pair comes along and picks up  your code they're going to know you're slack asses.  Maybe that makes it more then twice as hard?  The point is, pair programming is really good at reducing the amount of slop that you let into your code.  So come to think of it, maybe that being slopping part isn't really true about agile.  But, at least we still don't have to ever ship anything.

Hrm, but we do have those weekly iterations (that's another lie our iterations aren't weekly at the moment).  The thing about weekly or biweekly iterations is that they force you to be able to "ship" something at the end of every iteration.  I say "ship" with quotes around it because we all know that you're not really able to physically ship every iterations set of work to a client.  There is a boat load of stuff that needs to be done in order to actually ship.  You need to make a branch/tag in your source control provider.  You need to package up the software, burn it to a CD/DVD, prepare release notes, and a bunch of other stuff.  But you could ship something to somebody, even if at the very least it's to the QA department.  On non agile, or more correctly, non iterative projects that doesn't happen.  Come to think of it this iteration thing seems to put a lot of restrictions on the development team.  Let's outline a few of the things that are popping into my head:

- Forces development team to continually stay focused on meeting delivery dates, as in every week or two
- Forces development to communicate with one another constantly about the status of the iteration
- Puts a pretty rigid schedule in place for the development team
- Focuses the entire team on the work at hand, and what needs to be done to get it done by the end date
- Allows the product team to have visibility into what the development team is working on every day and what progress is being made

I'm not sure if everyone's agile process is like ours but it's probably the furthest thing from "cowboy coding", "sloppy", and it most certainly doesn't allow the development team do whatever the heck they want.  On the many non agile projects that I've been on in my career I had a lot more flexibility with how I spent my day.  As long as when that date hit down the line everything was done, I was good.  In many regards I had free reign to do whatever I wanted, I could code the thing however I wanted, and could have made a complete mess if I wanted. 

Working in an agile environment is much more rigorous and disciplined.  We have a very strict set of principles and practices that we follow.  We pair program, which means we have two people working on most production code, which leads to much less slop code, much less just get it done code, much less we're never going to look at this again code, much less who cares what it looks like the test passes code.  Our agile process defines a very rigid schedule for us that defines when, and in many cases how, we do everything we do.  

If anyone ever tells you that Agile is easy, that it's sloppy, that it lets the development team do whatever they want, they haven't been on an agile team doing it day in and day out.  It's hard.  It takes discipline.  It's draining.  It can be monotonous, invigorating, scary, frustrating, annoying, enjoyable, puzzling.  If you're doing it and it's "sloppy" you're not doing it.

In my post summarizing some of my thoughts from the latest Philly ALT.NET meeting on Planned Agility I alluded to the fact that the retrospective is a key component within our agile process.  Since it sounds like we have a slightly different flavor of retrospective then some others I thought I'd put together a post detailing how we run our retrospectives.

A couple times over the last few years our team had the pleasure of working with Jim Shore, author of a great agile book entitled The Art of Agile Development.  Jim introduced us to the style of retrospective that we're using "today".  While we've strayed away from the format I'm about to outlay a couple times, we've found that it provides us with a good foundation within which we can get to the root of the things that we want our retrospectives to uncover.

We typically start our retrospectives off by reminding people of the Prime Directive (should we?).  Once that bit of housekeeping is out of the way we try to get everyone involved by going around to everyone that is participating in the retrospective and have them answer a question.  The primary purpose is to get people engaged, and speaking as early as possible.  The moderator chooses the question, and a "fan favorite" seems to be "What do you hope to get out of this retrospective".

Next, we review our team objectives.  As I'll get to in a bit, we try and end each retrospective with a team objective.  By starting off each retrospective with a review of our objectives we ensure that we maintain focused on the objectives that we've defined.  During this time we evaluate how we're doing with each objective, whether or not they need to be altered, dropped, or given a kick start.  

After reviewing our objectives, we go into an information gathering session.  The purpose of this phase is to gather information from everyone in the team, and hopefully identify an item, or set of items that should be further addressed.  We use index cards for everything under the sun so it shouldn't be surprising that we use them for this process as well.  The way this stage works is that everyone on the team is given a handful of index cards and a pencil.  Each person reflects upon the last iteration (or whatever period of time it's been since our last retrospective), and identifies things that fit into one of the following topics:

• More
• Same
• Less
• Enjoyable
• Frustrating
• Puzzling

We usually take 5-10 minutes to allow everyone to write whatever pops into their head on their index cards at this point.  When I'm leading the retrospective I often encourage people to prepare ahead of time for this phase.  Once everyone has had enough time to write their cards, we go around and everyone reads the topic that their card goes within, and then gives a short overview of what is on their card.  Once everyone has had a chance to go we allow people to put up cards that they've thought of as everyone else was going around.  Once everyone is satisfied that they've gotten their major items identified on the board we begin the next phase.

Once all the data is collected on the board, we have everyone organize the cards into categories.  The categories are pretty loose and can be whatever anyone sees as fitting.  There is usually some debate at this stage as people discuss whether or not things really belong in category X instead of category Y.  Once this stage is complete, we review the categories that have been identified.  Often there is some slight re-organization at this point.  Once everyone is happy with our categorizations we do a quick vote to see what category people feel is the most important.  We then tally the votes and identify the item that is going to be the focal point for our next phase.

With our primary category (or sometimes item) in hand, we begin the discussion portion of our retrospective.  This can take the form of a fishbowl, roundtable discussion, or 5 why's.  The goal at this stage is to drill down into something within our team, process, work habits, communication, or whatever else that needs particular attention.  As part of this discussion a theme usually starts to emerge.  Once it does we identify it, and bring it to everyone's attention.  The primary theme typically turns into a new team objective.

As we go through our retrospectives we try to stay focused on things that we can do to help improve our team, and/or our software.  Since the human side of software is the most important and most problematic, you won't be surprised to hear that our objective usually has something to do with improving the dynamic between some members of our team.  We write our objectives on a wall in the development area for all to see with the idea being that if it's in everyone's face all the time we'll be more likely to meet our objectives.

I breezed through a lot of the details of our retrospectives above in order to get a reasonable summary up and out into the wild.  If you have questions about anything mentioned above or would like me to further expand upon anything mentioned drop me a comment.  If you haven't had a retrospective in a while, now is a great time to schedule one! :)

This past wednesday I attended the Philly ALT.NET user group meeting and got a chance to see Dave Laribee present on "Planned Agility".  Dave did a great job introducing how to do planning as part of your agile process.  Many people unfamiliar with agile think that planning isn't a part of the agile process, which of course is not the case. 

Planning happens at many levels within an agile process.  The primary types of planning that we do within our agile process are:

• Daily planning: Daily standup meetings start the day and help us figure out what we're going to be working on for the day.  We also evaluate where we are within the iteration and evaluate whether we're on schedule to deliver all the business value that has been agreed to, or whether we need to make small adjustments to meet our goals.
• Iteration planning: At the start of every iteration we have a meeting to discuss what business value the iteration is going to focus on, and then figure out what we can accomplish within the iteration
• Release planning: This is primarily done by our product owner and his team of business analysts.  A lot of discussions occur that help prioritize the features within a given iteration, as well as the scope of those features.  As work progresses within our iterations the release plan is adjusted to add and remove items.
• Product planning: Again primarily done by our product owner, this is the longer term planning that is done to lay our the longer term roadmap for the product, and to schedule the releases that are defined as part of the release planning.

Although it's not a planning activity per se, one of the other key things that often times affects our planning is our retrospectives.  Retrospectives may well be the most important activity within any agile process.  Without retrospectives we tend to fall into a rut, lose track of the problems and inefficiencies within our process, as well look past the ways that we could be improving our team, our software, and our code.  Our retrospectives often lead to planned activities and/or meetings that go into our next iteration planning session.   If you're not doing retrospectives, start.

Planning is an extremely important part of any agile process.  If you're doing agile without planning you're doing something wrong.  I suggest you try and catch Dave's presentation on "Planned Agility" at one of the upcoming conferences he's presenting at.

One of the best things about working in a software shop that has embraced the principles and practices of agile software development is the continued focus on what's really important.  Every week, our product manager and our business analysts are forced to think about what the most important piece of functionality is that can be added to our software.  I'm sure it's frustrating for them to have to, week after week, put things that seem relatively important on the back burner for those things that are most important. 

Focusing on the most important things can be frustrating, and it can lead to business trying to slip "easy" things into iterations whenever possible.  I believe this is a symptom of having to make so many concessions each and every week.  Since they never get everything they want, it's the "easy" things that are the most tempting to add.  However, I'm of the belief that "easy" is never good.  Doing something because it's easy, is doing something wrong.

It's no secret that a lot of software that is developed today is total crap.  It's loaded with feature upon feature.  It's hard to use, and doesn't do anything that helps the user kick ass.  Anybody can do "easy"....you don't want to be just anybody do you?

A couple days ago I posted about a nice video on InfoQ of Fred George talking about Appling Agile to Ruby.  The video on InfoQ was my first exposure to Fred, however, I recently came across his blog (via Jeremy and the ThoughtWorks feed).  Fred has a bunch of great content, such as:

• The Secret Assumption of Agile
• All Programmers are NOT Created Equal (People Topic)
• Masters, Journeymen, and Apprentices: Part 1
• Masters, Journeymen, and Apprentices: Part 2
• Masters, Journeymen, and Apprentices: Part 3
• Building a Better Programmer, Part 1
• Building a Better Programmer, Part 2
  

And some choice quotes:

Agile succeeds when you write code that is easy to change. The Secret Assumption of Agile

Okay, so it may not be true that all programmers are equal. But it is it approximately right? In my experience (many, many years)
on real projects with real delivery deadlines, there is an order of magnitude difference between programmers (that is, 10x difference). Even after throwing out the incompetent programmers (who produce zero), the really good programmers are ten times better than the really mediocre programmers. So even in approximate terms, all programmers are not equal. - All Programmers are NOT Created Equal (People Topic)

Over the weekend I listened to the latest .NET Rocks episode with John Lam.  I'm very keen on seeing and hearing about where the IronRuby project is going, so I found the interview very informational.  Although I enjoyed the entire show, one particular exchange between Carl and John stood out.  About half way through, Carl asked John whether he thought developers who don't do test driven development (TDD) and unit testing should be working in dynamic languages.  John made the wonderful assertion that he doesn't believe programmers that are not writing "unit tests" should be writing code in any language, regardless of whether the language is dynamic or statically typed.   Following that statement, John went on to mention that whether or not the code is test driven isn't nearly as important as whether the code has unit tests since the "test driven" approach is "extreme" and "dogmatic".  While I agree that the key is unit tests, I don't agree that a test driven approach is extreme or dogmatic.  In my opinion its the most pragmatic approach to writing code that is well designed and well tested.  The fact is, writing unit tests after the fact sucks!  Especially if "after the fact" is days or weeks after the real code is written.

When I first got started with TDD I didn't really understand what it was all about.  Rather than figure out what this magical red-green-refactor thing was all about I decided I'd just write some unit tests once I had my code written.  After a couple projects using this approach I started to realize that I really hated writing unit tests after the fact.  It was because of this hatred that I got started with test driven development.

Rather than testing being a pain in the ass task that I had to do when I finished writing my "real" code, testing became an instrumental step within the process of writing the "real" code.  Since I was using tests to figure out what code needed to be written, it became a vehicle by which I could design the code I was writing, rather than an afterthought.  I'm sure there are people out there that write great tests after the fact, however, I'm not one of them.  For me writing tests first isn't something extreme or dogmatic, it simple the only way I can write high quality code with solid test coverage without going nuts.  For me, writing unit tests after the fact sucks.  For me, driving my code with tests is a joy. 

I wonder how writing tests first could be "extreme" and "dogmatic" for others, but for me its pragmatic?

   

Over the weekend there was quit a flurry of heated discussion about the Composite UI Application Block (CAB) that was developed by the Patterns and Practices group at Microsoft.  It started with Ayende's " Why I don't like Patterns & Practices efforts " post, and continued with Sam On CAB and P&P and More on CAB and PAG .  Things continued with Ayende's Sam Gentile is mad at me , and On the CAB Again, and Be Silent they are Agile.  Not wanting to leave all the fun to Sam and Ayende, Hammet (of Castle fame) responded with The CAB way, and signs of community immaturity , Chris Holmes jumped in with his Ayende Bashes P&P and finally Adi offers a caution with his Caution: This blog contains personal opinions post.

From my perspective, there is a bit of complexity with CAB.  However, as some others have pointed out in the comments to the posts mentioned above, thats more due to the fact that CAB encompasses a couple different components that utilize patterns that not everyone who is starting out with CAB is comfortable with.  When you get past the initial hurdles, CAB can be broken down pretty simply.  It provides dependency injection so that the various components within your application can be loosely coupled and injected at runtime (similar to Windsor, Spring, etc).  It provides an eventing infrastructure that allows for the events and event handlers within your application to be loosely coupled (you might be able to think of it as a message bus for your UI), and it provides a number of infrastructure services that allow for your application to be broken up into a modules that can be used to create an application that can be dynamically composed (think MasterPages with a little bit of a plugin type architecture for building the UI at runtime).  For those starting out it requires familiarity with dependency injection, model view presenter or model view controller, loosely coupled events/commands, as well as an understanding of some of the requirements for a dynamically composed UI.  While it's not something I would use for every application I develop, I think it's a good fit for certain applications. For anything web related I'd steer clear of something architected in a similar fashion and go with MonoRail or if you can swing it Ruby on Rails. :)

Unfortunately, as I'm sure many in the chain of blog posts above would attest, blog posts aren't always the most effective way to carry out a discussion.  It doesn't help when those involved are a group of deeply passionate people who have strong feelings and opinions.  I think the primary points of each person's blog posts are important, relevant, and worth a continued discussion.  As Ayende points out, there are many things within Microsoft, as well as within the deliverables provided by P&P that seem to be more complex then they need to be.  As Sam states, we shouldn't lose sight of the fact that that P&P group is one of the few openly Agile groups within Microsoft.  I really believe they're trying to do the right things for customers, they're trying to do as many things as possible to make sure they're not developing in a vacuum, and that they're working on the things that their customers really need.  Are they perfect, absolutely not.  Are they trying, I think so.  Are the deliverables they offer for everyone, no.  Are their deliverables appropriate for some, yes.

I think Hammet brings up another important point for us all to think about regarding our community.  We have a lot to learn, and a lot of areas in which we can grow.  Insulting one another, isn't the way to go.  Letting our emotions get the best of us is something we should try and avoid.  However, at the same time we don't want to lose our passion and fire for what we believe in.  Let us remember that we're all part of the same community, and ultimately I think we're all after the same things.  When our emotions run high and we either say things that we wish we hadn't said, or state things that aren't exactly true, we need to take a step back and realize that sometimes our emotions do get the best of us.  But as far as I'm concerned, that's part of the process.  We're all human.

Over the last couple of days I've spoken with Sam a few times about what's unfolded.  For those of you who don't know, I have a unique perspective on the situation since I work with Sam.  He's a deeply passionate individual who loves technology.  He has an unquenchable thirst for new information as our printers at work can attest to.  He genuinely wants to help the .NET community, and one of the ways that he's done that is in pointing out some inaccuracies in Ayende's post on CAB and P&P.  Based on conversations Sam and I have had in the past, I think besides some of the inaccuracies on how CAB was developed, Sam agrees with most (or maybe just some) of what Ayende has said. As Jeremy pointed out, bringing up things that were said years ago isn't the way to address any problems you might have with Sam on how he voiced his concerns.  Sam is a perfectionist, and will welcome any positive and constructive feedback you can provide him.  My constructive feedback is to not let your emotions lead you to say things you'll regret the next day.  But I'd also suggest you keep your passion, and keep doing the things that make you you.  At the end of the day the most important things for all of us is to be happy.  Don't let what's happened in the past get you down.  We live and we learn.  All we can hope for is that we better ourselves over time.  Let's leave the personal attacks to the Java community (just kidding any of you Java peeps out there), and hope we can make the .NET developer community a better place for those of us who care about producing great software.

Brian Marick has a great post where he talks about some values that he thinks agile teams should invest in.  While they're not part of the original Agile Manifesto, I agree with Brian (afterall who am I to disagree :)) that they're key to helping agile teams be successful.

• Skill
• Discipline
• Ease
• Joy

On a related note, I also recommend you take Sam's advice about "Being Real and Communicating".

Most of the stories I hear about using an agile/iterative development "methodology" are from those in the consulting world.  It's nice to hear about people in product organizations, such as Adobe, using agile/iterative development successfully.  I've heard a number of people say it isn't possible due to the nature of product organizations, it sounds like the Photoshop development team might disagree.  They switched to a more agile approach and cut their bug count by approximately 1/3, ended up with a higher quality product with plenty of features, and ended up working fewer nights and weekends.

This past week, while working on a couple updates to some existing code I had to update the corresponding unit tests to ensure the stuff I added didn't cause massive problems.  As I started exploring the unit tests for the existing functionality my head started to spin.  Either I was going nuts, or the tests were less than ideal.  After examining the code for a little while I started to see why my brain wasn't working so well when it came to examining this particular chunk of code.  While the tests were testing the functionlality that they needed to, they weren't nearly as readable as they could have been.  Some of this is a result of attempting to reduce the amount of duplication within the test class.  You see, several things needed to assert the same things, so they were seperated out into a method that you could pass some stuff to, and it would magically assert everything that you could ever want asserted upon the so said "stuff".  While nice in concept, it led to code that was very hard to follow.  In order to see what an individual test was doing I needed to jump around to 3 or 4 methods that contained the asserts.  Once I finally tracked down the asserts that were being used I was able to see what was expected.  It just so happens that I could also see that we were testing the same thing 3 or 4 different times in an individual test.  For good measure, we also appeared to test the same exact logic in multiple tests!  All of this has led me to the following conclusions:

• You should prefer readability over well factored code for unit tests.  If you need to jump all around to figure out what you're testing something's wrong.  If duplication helps with readability, duplicate away!
  
• Make sure your unit tests are only testing one thing. 
  
• Make sure you don't re-test the same thing over and over and over again.  Taken with the above point, you should begin to see that you shouldn't have that many tests that need to assert the same thing, if you do, you probably have multiple tests testing the same thing.
• Only tests one class per unit test class.  If you're testing ClassA and it leverages logic within ClassB and ClassC, don't test ClassB and ClassC's logic within your test for ClassA.  Create seperate classes for ClassB and ClassC.  Trying to test them all in one place will lead to unit tests that are too hard to read and doing too much.  This is particularly hard to keep in mind since often times as part of the process of doing test driven development ClassB and ClassC might be created.  It's tempting to leave the existing tests that test the logic that used to be in ClassA but has since been moved to ClassB and ClassC within ClassA's tests.  Don't do it, move the tests to the suite of tests that you're creating for ClassB and ClassC.  You'll thank yourself when you come back to change that logic, since it will be where you expect, rather than in a test class for a related class.

I've gotten distracted by Lilo and Stitch so I'm going to stop with my conclusions there, and go play some indoor basketball with my son who is throwing a basketball all over our house. :)

[Disclaimer: It should be noted that I'm pretty sure the code I was looking at was code I wrote...whoops :(]

With all the talk of being Agile, one has to wonder if anyone cares about shipping.  We focus a lot on the methodologies and practices that go into creating software, but we don't talk all that much about shipping.  Software doesn't exist until it's shipped.  No money is made from a great piece of software that isn't available.  No money is made from having a really kick ass Agile process if you don't ship.  It's not all about money, but without money we don't get to "play another game of pinball".  As Sam reminded me today, it's all about getting that next game.  It's all about shipping.

Those of us living in the world of software need to remember why we're in this business.  It's not about process, it's not about methodology, it's not about how Agile you are. It's about creating kick ass software that users love.  Users can't fall in love with something that isn't shipped.

In this day and age Agile seems to be where it's at.  If you're Agile, you're good.  If you're Agile, you have no bugs.  If you're Agile, you respond quickly to change.  If you're Agile, you'll build what your customer wants. 

It's easy to say you're Agile.  It's even easy to think you're Agile.  I wonder though, are you really Agile?  Are we Agile?  Am I Agile?  What does it even mean to "be Agile"?  Is it even possible?

What does it take?  Do you have what it takes?