Getting Some Real Offshore Experience

In a former post (see "Offshore Development Woes"), I wrote some personal conclusions from reading Martin Fowler on "Using an Agile Software Process with Offshore Development". It was a summary of what I got from Mr. Fowler's experiences. Interestingly, though I have worked with many developers from many Asian countries over the years, I have never actually worked with an offshore software development company performing development for hire under my direction. It's easy to be an armchair critic.

But that's changed, at least a little. Faced with a recent influx of development projects and a lack of staff to get them done in a timely manner, I decided to develop a relationship with a firm based in India. Being the cautious type I am, our initial foray into this arrangement was for a limited time and scope -- I really wanted to see how well it would work out. And I was especially intrigued given Mr. Fowler's experiences because we too are an agile development shop. I had misgivings based on the rapidity of our development and the time and language differences. But the hourly rates proffered were downright seductive. I figured that at the $25 per hour rate we were being charged, we could afford to lose some time to these difficulties and still come out ahead. And I really want it to work.

It's now been 7 weeks into this trial and I guess I have mixed results to report. First, as Mr. Fowler observed, there is no real cost savings in spite of the seductive hourly rate. Between the language barriers, the time differences, things just plain take longer. A lot longer. And that costs. It's not like the developer in India isn't charging me for those differences. We use email and Skype to get our communication done since written English at least removes accents from the equation. And our developers in India work odd hours, so in spite of the 9.5 hour time difference, I can usually talk to them throughout the morning.

But I've now worked with the group on three development tasks. The initial effort was to get the existing project development infrastructure (Eclipse, Tomcat, CVS, MySQL, and the project code) up and running. I have to say, I was impressed with how quickly and smoothly that happened. It showed that the Indian team was well acquainted with these tools. So we moved on to actual development tasks. I made an effort to be more clear and concise than I usually am in providing task writeups. We had several email exchanges asking for clarification and I got my first code delivery in about a week. I was very disappointed. The code was no where near complete, was undocumented, did not follow style guidelines, and was generally unproffesional (a java package name of "jesus.xml"?). But the real problem was the lack of some basic computer science -- a huge switch construct instead of arrays; a single large method instead of encapsulation, and so on. I rewrote the code and sent it back with further instructions on what I was looking for.

To make a long story short(er), this seems to be the pattern -- delivered code is under-par, I rewrite it and return it, and they then follow my pattern. This put the first several tasks way over budget.

We did make a complaint to the management team. Their solution was to add another resource (a more senior programmer) to the team. At their own cost (to their credit). But now, every detail is being questioned and designed out in advance. We've lost our agile approach. And the costs are higher still, since I spend so much time trying figure out the answers to their questions.

So, in all, this effort has proven to be more costly than we had hoped and it pushed our project costs over budget. It may be that a different team from India could perform better, or that if we developed using a traditional "waterfall" life cycle, things would be better. But this has not been a success for us.

Offshore Development Woes

In his excellent article: "Using an Agile Software Process with Offshore Development", Martin Fowler summarizes his experiences in using offshore development for large projects. While his particular emphasis is on agile versus waterfall development approaches and the specific twists for offshore agile development, he makes a startling conclusion regarding offshore development: "Certainly anyone doing it because they think they'll get cost savings similar to the rate differences is seriously deluding themselves. "

This really hit home for me. We've been seriously examining our rates, partly because of our perception of downward pressure caused by the offshore "threat". But as I looked at it more, it seemed to me that there were several misperceptions going on here. First, it turns out that offshore arrangements we're competing with are under a contract for a minimum number of hours. Well, I'm certainly willing to offer a generous discount for a commitment like that! But more importantly, there is the hourly rate perception.

Just because the hourly rate is lower does not mean the project will be cheaper. I've always felt that way, but Martin Fowler supports it with evidence. Fact is, offshore development carries a large communications and distribution overhead that offsets the lower hourly rate.

He also says that "anyone who thinks that onshore developers will triumph because they are more skilled is very wrong. We've found that we can hire just as talented developers in India as we can in North America and Europe." And this has been my experience as well. The talent available there is excellent, though (IMHO) not better than we have here.

So, at the moment, it would seem to me to be a question, not of savings or talent, but of industry knowledge. The real savings should come about by using developers skilled and knowledgeable, or at least conversant in the industry area the development is supporting.

I say, the playing field is level, competition is good, let's sharpen our tools and have at it...

Estimating Software Development

I'm an excellent software architect, developer, project manager, and even (as needed) sales person. But I can't estimate a project development effort to save my life. I waaay underestimate. Every time. I can lose money faster than a gambler in Las Vegas! So I figure I need to study the science (if such there is!) of estimating. In the past I've just thrown numbers into MS project and hoped for the best.

I was initially turned on to this idea of a systematic approach to estimating by one of the smarter folks I know, Norbert Winklereth, formerly of Omnimark and Stilo. He espouses an approach I had never heard of -- Wide Band Delphi Blind (WBDB). Being a believer in Agile methods, this appeals to me. I started reading up on it and I like what I see. It depends on several people looking at the whole problem (Wide Band), to develop a list of tasks and an estimate for each (The Oracle of Delphi) operating anonymously to reduce or eliminate political influence or pressure, (Blind). There's a fair writeup of it HERE.

There are a lot of parts to this, but the essence of it is to get several people working together to bring their collective expertise and experience to the issue of estimating an effort. And, it can be used to estimate a number of things, not just cost, but anything to which we can apply a unit of measure. One thing I began to see early on is that estimating the effort of a project and estimating the schedule are two completely different, though related things.

Then I read an article by Joel Spolsky (Joel on Software) on something he calls "Evidence Based Scheduling." Joel argues for (and, indeed, has a software product based on) creating schedules based on historical performance data (from time sheets or other records) combined with Monte-Carlo simultations. Again, recognizing the difference between effort and schedule, Joel makes a point that any task planned for more than 16 hours is not going to work and needs to be further factored. I think the same thing applies to estimating effort -- break it down into units of not more than 16 hours.

Another factor that Joel recognizes is the relative historical accuracy of an individual as an estimator - what he calls velocity, the ratio of an individual's estimate to the actual. A perfect estimator would always have a velocity of 1. A person's history of these ratios is his velocity history and is a factor in the Monte-Carlo simulation.

So, I'm thinking that a blend of these techniques may represent a reasonable estimating approach. We need to assemble a blind group, and a project leader who has and continues to build a velocity history for each blind participant. We need some Monte-Carlo simulation software, and maybe a tool to organize all this.

More to come

Agile and Budgeting

Scott Ambler has an excellent discussion of the relationship between, resources, and schedule in the "Iron Triangle" of software development and the need for flexibility in at least one of these areas to ensure project success. Scott concludes by outlining several scenarios in which one or more of the sides of the triangle can vary.

In my experience, the budget for a project is usually fixed, or at least proscribed to a certain upper boundary. So there are really only two sides of the triangle that can vary -- the scope and the schedule. Of these two, I find that the schedule is usually less flexible that the scope -- often there is a market-driven date establishing an outer boundary on the development process.

Which leaves us with the scope of the project as the only negotiable or variable side of the triangle. This leaves the development manager in a somewhat difficult position -- how to satisfy the user community (who's focus is on features) and also the client bursar (who's focus is on budget and schedule). The key to this is to develop the most important features first and leave the less important features for the end of development. This, in turn, requires a frank and realistic discussion of feature priorities. So, at the outset of a project, perhaps the most important thing the development manager can do is to get a solid list of the features and their priority.

Back to the GWU Summer Publishing Institute...

So, getting back to the GWU Summer Publishing Institute, and the topic on which I need to speak, "Agile vs. Traditional: Methods for Building a Software Infrastructure."

It seems to me that, after exploring what Agile Publishing might mean, it's clear that an Agile Publisher needs a software infrastructure that supports the key tenets. I take that to mean a content management and publishing infrastructure that supports the tenets of Agile Publishing:

• Customer satisfaction by rapid, continuous delivery of useful product
• Working product delivered frequently
• Working product is the principal measure of progress
• Even late changes in requirements are welcomed
• Close, daily, cooperation between business people and developers
• Face-to-face conversation is the best form of communication
• Projects are built around motivated individuals, who should be trusted
• Continuous attention to technical excellence and good design
• Simplicity
• Self-organizing teams
• Regular adaptation to changing circumstances

So what kind of framework is that? I'm envisioning a content management and publishing system that easily pushes out releases on a frequent basis, allows content and structural changes to be readily made, enhances communication between participants, enables remote meetings, provides templates for good design, has easy-to use process and product review tools, enhances workflow with loose rules, and can be easily adapted to changes.

This goes a ways beyond a content management system, or any publishing system on the market today because it demands flexibility and adds project collaboration tools.

In "The Emerging Art of Agile Publishing", XML.COM, 3/8/2006, Michael Fitzgerald makes some specific suggestions about how to be an Agile Publisher:

• "Build real trust through constant, informal communication
• Interleave work processes between writers, editors
• Share work openly, don't store it in secret silos
• Store source files in an online repository and give all members of the team access to it
• Agree on and share tools to reduce the waste of format conversions
• Carve up your work into small, easily consumed and exchanged pieces
• Keep track of what you are doing daily with some other suitable tool
• Publish your work early and often, charge a little something and get free reviews
• Let Herb do things his own way, but not on your team"

I would add to that, "Keep it simple".

So what software is needed? First and formost, a communications tool. Especially in these days of telecommuting, getting face-time is hard. I like instant messaging and the Skype product in particular. Skype has a secure chat and telephony in one kit. Second, a content management system that will permit easy and frequent releases of product. There are many of these, including several open source ones. Third, open project management. The basecamp.com approach seems nice as does the offering from project.net. I'm personally in favor of an open project document repository for sharing project related materials. This could also be used for openly managing project management materials, such as an MS project plan.

Agile Publishing -- Continuous attention to excellence, good design, and simplicity

Principles Number 8 and 9 of the Agile Software Development approach . What do these mean for an Agile Publisher?

As with some of the previous principles, this is partly just good business and I feel these are all closely related. Simplicity => Good Design => Excellence.

Especially when developing a complex product with many content components, it is easy to design an overly complex, difficult to build and maintain product. Simplicity is the name of the game here. Also, good design -- keeping sight of both the immediate need and the growth room and designing with these in mind.

Agile Publishing -- Projects are built around motivated individuals, who should be trusted

Number 7 of the Agile Software Development approach . What does this mean for an Agile Publisher?

It's all about trust. This also could be said of every project everywhere. There's nothing different here for an Agile publisher than for any other development effort. Michael Fitzgerald ("The Emerging Art of Agile Publishing", March 8, 2006, XML.COM) posits these tenets:

• People must be trusted
• Fewer but more competent people are needed
  
• Organizations must live with the decisions developers make

These are key points. Trust inspires dedication, dedication reduces numbers of people. But (I hear you cry), organizations must live with the decisions developers make? Hard to swallow, that. But remember, a key point of the Agile approach is that the development team includes stakeholders. It is partly their role to ensure that the decisions the developers make are in the best interests of the product and the client. A better product will come of it.

Agile Publishing -- Close, daily, cooperation between business people and developers and Face-to-face conversation is the best form of communication

Close, daily, cooperation between business people and developers and face-to-face conversation is the best form of communication -- Principles Number 5 and 6 of the Agile Software Development approach . What do these mean for an Agile Publisher?

It's all about communication. This could be said of every project everywhere. There's nothing different here for an Agile publisher than for any other development effort. Michael Fitzgerald ("The Emerging Art of Agile Publishing", March 8, 2006, XML.COM) posits these tenets:

• The culture of the organization must be supportive of negotiation
• Organizations need to have an environment that facilitates rapid communication between team members

Bravo.

Agile Publishing - Even late changes in requirements are welcomed

Even late changes in requirements are welcomed -- Principle Number 4 of the Agile Software Development approach . What does this mean for an Agile Publisher?

As the publishing world changes to include more and diverse assets in a publication, we introduce the possibility (or even probability) that changes will occur during the product development life-cycle. These could be as simple as changes in artwork or as complex as an overhaul in related assets.

One of the significant features of the iterative nature of Agile development is that the stakeholders (client reps) can see if the product is taking shape as was expected. If not, changes are not only possible, but encouraged, at just about any point in the development cycle. This same benefit obtains for changes to the underlying business model -- should the market conditions for the product change, we can respond to these changes.

Retaining this ability to accept changes to the requirements, no matter when they occur in the development cycle is a pre-eminant feature of Agile and applies equally to Agile publishing as it does to Agile software development.

Agile Software Development v. Traditional "Waterfall"

I've been involved in two projects recently in which I got a first hand opportunity to compare the results of software development under Agile and under the traditional "waterfall" method. The two projects were not really similar in any way, but they still bear some scrutiny for the results of the two different approaches.

The project done using a tradition (or waterfall) methodology (which I'll call "Project W") began with a requirements gathering phase in July/2004. This was followed by an architecture and design phase, programming, testing and acceptance testing. We will (hopefully) deploy this system to customers in June, 2007, nearly three years after the project started.

The project done using Agile (which I'll call "Project A") was started in August, 2006 and was delivered and live in production in December 1006, around 5 months after the start.

Now, there are some very big differences in these two projects, including the complexity of the underlying data models and in the overall software architecture. However, it is interesting to note that a big part of the 3-year time to deliver on Project W came from two factors: redesign of system components that were found to be faulty during programming, and under-developed requirements. Especially in the past six months, we have spent a significant amount of developer resources adding/changing/deleting user features long after the requirement were solidified. In contrast, on Project A, at the point where the software was ready for testing, there were few feature changes.

Why? A significant aspect of Agile is that the client/user/stakeholder "sees" working versions of the software early on. So if there are any mis-understood requirements, these are found and fixed very early in the development life-cycle. Also, in Agile there is a much more limited design
process, relying instead on developing working code. As a result, any design errors were uncovered earlier in the cycle.

An interesting thing happened on Project A -- we exceeded the budgeted costs. Although this was conveyed to the customer as it happened, it still caused a great deal of consternation. However, and in spite of that issue, I view the project as a huge success. Had we followed a "waterfall" paradigm, it would likely have cost the same or more and would have taken longer to complete. So although we missed our target budget, we delivered a market-ready product in five months, enabling the customer to begin generating revenues quickly.

I'm totally convinced that Agile is superior for the kinds of projects on which I tend to work.