A New Kind of Content Management

I've been enamored with classification of content through my entire career (which is certainly long enough). Early on, I was working on library automation systems and in particular with Library of Congress Subject Headings (LCSH) data and I always thought classification of documents and other content types was a panacea we should try to achieve. Librarians used these and other tools as finding aids to locating specific information, usually books.

Along came the internet and the amount of information available grew exponentially. Search engines provide keyword access to text-base content. Publishing was redefined to include anyone putting information out on the internet. Published content became a thing to be managed with buzzwords like single-sourcing, XML, multimedia, syndication, blogs, Wikis, and so on.

Well, the world is very different now and with everything from books and documentation to audio and video as well as thought streams all published on the web, finding a particular bit of information can be a nightmare. Even traditional textual content is hard to find in the vast archive that is the internet.

So how do we find things now? What are our modern finding aids starting to look like?

Some very cool new technologies have come along recently: folksonomies and mashups, to name two.

Folksonomies in particular offer a new approach to an old problem -- how to capture the "aboutness" of content. Classifying content is an age-old process that has spawned many fields including taxonomies and indexing. These, in turn, have created an arcane set of rules and procedures wherein the maintenance of the indexes or thesauri become big efforts on their own -- at times larger than the content management effort they are supporting.

Mashups are a cool way to combine content from multiple sources into a single presentation. Kind of like federated search and portals in a social-networking environment.

I'll be exploring these more over the next several posts.

Offshore Development Woes

In his excellent article: "Using an Agile Software Process with Offshore Development", Martin Fowler summarizes his experiences in using offshore development for large projects. While his particular emphasis is on agile versus waterfall development approaches and the specific twists for offshore agile development, he makes a startling conclusion regarding offshore development: "Certainly anyone doing it because they think they'll get cost savings similar to the rate differences is seriously deluding themselves. "

This really hit home for me. We've been seriously examining our rates, partly because of our perception of downward pressure caused by the offshore "threat". But as I looked at it more, it seemed to me that there were several misperceptions going on here. First, it turns out that offshore arrangements we're competing with are under a contract for a minimum number of hours. Well, I'm certainly willing to offer a generous discount for a commitment like that! But more importantly, there is the hourly rate perception.

Just because the hourly rate is lower does not mean the project will be cheaper. I've always felt that way, but Martin Fowler supports it with evidence. Fact is, offshore development carries a large communications and distribution overhead that offsets the lower hourly rate.

He also says that "anyone who thinks that onshore developers will triumph because they are more skilled is very wrong. We've found that we can hire just as talented developers in India as we can in North America and Europe." And this has been my experience as well. The talent available there is excellent, though (IMHO) not better than we have here.

So, at the moment, it would seem to me to be a question, not of savings or talent, but of industry knowledge. The real savings should come about by using developers skilled and knowledgeable, or at least conversant in the industry area the development is supporting.

I say, the playing field is level, competition is good, let's sharpen our tools and have at it...

Estimating Software Development

I'm an excellent software architect, developer, project manager, and even (as needed) sales person. But I can't estimate a project development effort to save my life. I waaay underestimate. Every time. I can lose money faster than a gambler in Las Vegas! So I figure I need to study the science (if such there is!) of estimating. In the past I've just thrown numbers into MS project and hoped for the best.

I was initially turned on to this idea of a systematic approach to estimating by one of the smarter folks I know, Norbert Winklereth, formerly of Omnimark and Stilo. He espouses an approach I had never heard of -- Wide Band Delphi Blind (WBDB). Being a believer in Agile methods, this appeals to me. I started reading up on it and I like what I see. It depends on several people looking at the whole problem (Wide Band), to develop a list of tasks and an estimate for each (The Oracle of Delphi) operating anonymously to reduce or eliminate political influence or pressure, (Blind). There's a fair writeup of it HERE.

There are a lot of parts to this, but the essence of it is to get several people working together to bring their collective expertise and experience to the issue of estimating an effort. And, it can be used to estimate a number of things, not just cost, but anything to which we can apply a unit of measure. One thing I began to see early on is that estimating the effort of a project and estimating the schedule are two completely different, though related things.

Then I read an article by Joel Spolsky (Joel on Software) on something he calls "Evidence Based Scheduling." Joel argues for (and, indeed, has a software product based on) creating schedules based on historical performance data (from time sheets or other records) combined with Monte-Carlo simultations. Again, recognizing the difference between effort and schedule, Joel makes a point that any task planned for more than 16 hours is not going to work and needs to be further factored. I think the same thing applies to estimating effort -- break it down into units of not more than 16 hours.

Another factor that Joel recognizes is the relative historical accuracy of an individual as an estimator - what he calls velocity, the ratio of an individual's estimate to the actual. A perfect estimator would always have a velocity of 1. A person's history of these ratios is his velocity history and is a factor in the Monte-Carlo simulation.

So, I'm thinking that a blend of these techniques may represent a reasonable estimating approach. We need to assemble a blind group, and a project leader who has and continues to build a velocity history for each blind participant. We need some Monte-Carlo simulation software, and maybe a tool to organize all this.

More to come

Agile and Budgeting

Scott Ambler has an excellent discussion of the relationship between, resources, and schedule in the "Iron Triangle" of software development and the need for flexibility in at least one of these areas to ensure project success. Scott concludes by outlining several scenarios in which one or more of the sides of the triangle can vary.

In my experience, the budget for a project is usually fixed, or at least proscribed to a certain upper boundary. So there are really only two sides of the triangle that can vary -- the scope and the schedule. Of these two, I find that the schedule is usually less flexible that the scope -- often there is a market-driven date establishing an outer boundary on the development process.

Which leaves us with the scope of the project as the only negotiable or variable side of the triangle. This leaves the development manager in a somewhat difficult position -- how to satisfy the user community (who's focus is on features) and also the client bursar (who's focus is on budget and schedule). The key to this is to develop the most important features first and leave the less important features for the end of development. This, in turn, requires a frank and realistic discussion of feature priorities. So, at the outset of a project, perhaps the most important thing the development manager can do is to get a solid list of the features and their priority.