XML: Implications and Opportunities

Everybody's heard of XML and all the hype that comes with it. But what is XML? And what does it mean for publishers?

Often, content is authored, edited and published using a markup form that only controls style and presentation and not meaning. Examples of this are word processor files and web pages. XML offers the chance to capture the meaning of the content in the markup. Instead of this:

Allen, Thomas B. Vanishing Wildlife of North America. Washington, D.C.: National Geographic Society, 1974.

We have this:

<citation>
<author>Allen, Thomas B.</author>
<title>Vanishing Wildlife of North America</title>
<publisher>
<name>National Geographic Society</name>
<place>Washington, D.C.</place>
<date>1974</date>
</publisher>
</citation>

From that XML we could generate the same citation or any of a number of other formats. But, with this XML, we can also find our content with searches such as "author=Allen" and so on.

The key to understanding XML and it's implications for publishers is "Content Use and Reuse." Let's look at a scenario from a project on which I'm currently working.

Current awareness (news) content is published daily. It was originally written in a proprietary markup format, for which there were lots of in-house custom tools. All the editorial staff know the markup language. New hires have a long learning curve. We replaced the proprietary markup with an XML form. We hid the XML behind a word-processor interface and publish in a variety of forms (old and new) from the XML. This is feasible because of the open standards basis of XML and the freely available tools for managing XML. But really, it's just economics. Saved money. A publisher embarking on this kind of restructuring can use any of a number of standardized content models as a starting point: ODF, DocBook, DITA, etc.

The real opportunity comes in the new ways the content can be extended and also reused. More can be done to describe the content, down to a very granular level. The XML form of the content can contain not only the content, but metadata describing the content. This might not appear in any one published version, but can be used for searching and linking applications. It can be published in many ways on different media through simple reformatting.

Semantic markup is the next big step. We capture not only the metadata, but also describe the "aboutness" of the content. From simple keywords to linked thesauri, to external, linked ontologies, we can capture many levels of domain meaning for the content.

Back to the GWU Summer Publishing Institute...

So, getting back to the GWU Summer Publishing Institute, and the topic on which I need to speak, "Agile vs. Traditional: Methods for Building a Software Infrastructure."

It seems to me that, after exploring what Agile Publishing might mean, it's clear that an Agile Publisher needs a software infrastructure that supports the key tenets. I take that to mean a content management and publishing infrastructure that supports the tenets of Agile Publishing:

• Customer satisfaction by rapid, continuous delivery of useful product
• Working product delivered frequently
• Working product is the principal measure of progress
• Even late changes in requirements are welcomed
• Close, daily, cooperation between business people and developers
• Face-to-face conversation is the best form of communication
• Projects are built around motivated individuals, who should be trusted
• Continuous attention to technical excellence and good design
• Simplicity
• Self-organizing teams
• Regular adaptation to changing circumstances

So what kind of framework is that? I'm envisioning a content management and publishing system that easily pushes out releases on a frequent basis, allows content and structural changes to be readily made, enhances communication between participants, enables remote meetings, provides templates for good design, has easy-to use process and product review tools, enhances workflow with loose rules, and can be easily adapted to changes.

This goes a ways beyond a content management system, or any publishing system on the market today because it demands flexibility and adds project collaboration tools.

In "The Emerging Art of Agile Publishing", XML.COM, 3/8/2006, Michael Fitzgerald makes some specific suggestions about how to be an Agile Publisher:

• "Build real trust through constant, informal communication
• Interleave work processes between writers, editors
• Share work openly, don't store it in secret silos
• Store source files in an online repository and give all members of the team access to it
• Agree on and share tools to reduce the waste of format conversions
• Carve up your work into small, easily consumed and exchanged pieces
• Keep track of what you are doing daily with some other suitable tool
• Publish your work early and often, charge a little something and get free reviews
• Let Herb do things his own way, but not on your team"

I would add to that, "Keep it simple".

So what software is needed? First and formost, a communications tool. Especially in these days of telecommuting, getting face-time is hard. I like instant messaging and the Skype product in particular. Skype has a secure chat and telephony in one kit. Second, a content management system that will permit easy and frequent releases of product. There are many of these, including several open source ones. Third, open project management. The basecamp.com approach seems nice as does the offering from project.net. I'm personally in favor of an open project document repository for sharing project related materials. This could also be used for openly managing project management materials, such as an MS project plan.

Agile Publishing -- Continuous attention to excellence, good design, and simplicity

Principles Number 8 and 9 of the Agile Software Development approach . What do these mean for an Agile Publisher?

As with some of the previous principles, this is partly just good business and I feel these are all closely related. Simplicity => Good Design => Excellence.

Especially when developing a complex product with many content components, it is easy to design an overly complex, difficult to build and maintain product. Simplicity is the name of the game here. Also, good design -- keeping sight of both the immediate need and the growth room and designing with these in mind.

Agile Publishing -- Projects are built around motivated individuals, who should be trusted

Number 7 of the Agile Software Development approach . What does this mean for an Agile Publisher?

It's all about trust. This also could be said of every project everywhere. There's nothing different here for an Agile publisher than for any other development effort. Michael Fitzgerald ("The Emerging Art of Agile Publishing", March 8, 2006, XML.COM) posits these tenets:

• People must be trusted
• Fewer but more competent people are needed
  
• Organizations must live with the decisions developers make

These are key points. Trust inspires dedication, dedication reduces numbers of people. But (I hear you cry), organizations must live with the decisions developers make? Hard to swallow, that. But remember, a key point of the Agile approach is that the development team includes stakeholders. It is partly their role to ensure that the decisions the developers make are in the best interests of the product and the client. A better product will come of it.

Agile Publishing -- Close, daily, cooperation between business people and developers and Face-to-face conversation is the best form of communication

Close, daily, cooperation between business people and developers and face-to-face conversation is the best form of communication -- Principles Number 5 and 6 of the Agile Software Development approach . What do these mean for an Agile Publisher?

It's all about communication. This could be said of every project everywhere. There's nothing different here for an Agile publisher than for any other development effort. Michael Fitzgerald ("The Emerging Art of Agile Publishing", March 8, 2006, XML.COM) posits these tenets:

• The culture of the organization must be supportive of negotiation
• Organizations need to have an environment that facilitates rapid communication between team members

Bravo.

Agile Publishing - Even late changes in requirements are welcomed

Even late changes in requirements are welcomed -- Principle Number 4 of the Agile Software Development approach . What does this mean for an Agile Publisher?

As the publishing world changes to include more and diverse assets in a publication, we introduce the possibility (or even probability) that changes will occur during the product development life-cycle. These could be as simple as changes in artwork or as complex as an overhaul in related assets.

One of the significant features of the iterative nature of Agile development is that the stakeholders (client reps) can see if the product is taking shape as was expected. If not, changes are not only possible, but encouraged, at just about any point in the development cycle. This same benefit obtains for changes to the underlying business model -- should the market conditions for the product change, we can respond to these changes.

Retaining this ability to accept changes to the requirements, no matter when they occur in the development cycle is a pre-eminant feature of Agile and applies equally to Agile publishing as it does to Agile software development.

Agile Publishing -- Working product is the principal measure of progress

Working software is the principal measure of progress -- Principle Number 3 of the Agile Software Development approach (http://en.wikipedia.org/wiki/Agile_software_development). Let's change this to "working product is the principal measure of progress" and explore what this might mean for an Agile Publisher.

Norbert Winklareth, one of the smartest people I know, asserts that the measure of value for which customers pay for is not any traditional metric of development but features:

1. Value => Functionality That Works
2. Functionality => Set of Features
3. Therefore: "Features are the true measure of development"

It would seem that the same principle can apply to a publishing project -- a publication has, among other things, a set of features. In software, features are specific characteristics, such as a search function or an RSS feed. In publishing, features could be harder to describe. In essence, these are the characteristics of a product for which customers pay. As a practical matter, features could be publication components (or assets) for example, written text, artwork, multi-media software functions, summaries or abstracts, and so on. Each of these can be considered as a feature of the published product.

Deciding on the priority of the features of our product is important here. By developing the most important features first, we can plan the creation of our product in stages (or iterations) in which we get our most desirable or most important features built first. The result is that we have a working, functional product at the end of each iteration that could be considered a final product if need be. It certainly permits us to respond to budgetary or other outside business pressures and still end up with a publishing product.

It also provides us with a true metric for progress. As features are developed, we can measure that easily against our plan for the product. We then know how well our project is fairing.

Agile Publishing -- Working Product Delivered Frequently

Working software is delivered frequently (weeks rather than months) -- Principle Number 2 of the Agile Software Development approach (http://en.wikipedia.org/wiki/Agile_software_development). What does this mean for an Agile Publisher?

One of the key issues that arises in traditional software development is that the client's vision of the features and functionality of a product are different from the developers' vision. Another is the inability of the stakeholders to control development against their budget. These two issues are at least partially addressed through the use of Agile's frequent delivery of working software principle. Specifically, by getting frequent releases, clients can see for themselves how conformant the developers' vision is to their own. Also, and especially when combined with another concept -- planning for priority features in the early releases, should the development cost begin to approach the project budget, the project can potentially be terminated early with much of the priority feature set in working order. Powerful stuff.

Developing a publishing product can follow a similar flow to software development, one in which iterations of content are created and reviewed. This is especially true for products with content derived from many sources. Following a similar flow of development, we can exert the same control -- create iterations of working versions of the product and with more important features (included content sources perhaps) developed first.

In the realm of electronic publishing, it is almost always true that a product is derived from many sources and I've seen many publishing efforts held up in order to include a particular content asset. I wonder if that asset was important enough to the product to incur the delayed release and resulting revenue.

Agile Publishing - Continuous Delivery of Working Product

Customer satisfaction by rapid, continuous delivery of useful software -- Principle Number 1 of the Agile Software Development approach (http://en.wikipedia.org/wiki/Agile_software_development). What does it mean for an Agile Publisher?

To my mind, this is one of the most important pieces of Agile. This is what ensures that what the customer envisions is what is being built and that if not, corrections/realignment can be done early and at the least cost. It has some secondary advantages as well: regularly tracking progress against a budget; early start on customer training; and easier collaboration with client/stakeholders, to name a few.

What does this mean to electronic publishers? Publishing describes a large, complex industry that delivers information to a user community. There are many forms of electronic publishing: newsletters, online books, references, electronic databases, and so on. There is a commonality to software development in that published products (or assets) go through a development/authoring process, a QA (or editing) process and delivery.

An Agile Publisher would apply the principle of "rapid, continuous delivery" to each electronic product. In practice this could mean author submissions of partial manuscripts, developer delivery of database subsets, and so forth. Any of these early and frequent data feeds provides a foundation for "rapid, continuous delivery", which the permits the goals of early and ongoing review and realignment, budget tracking, etc.

But, to take advantage of this, the electronic publisher must also have a deployment infrastructure that will create a working version of this electronic asset. In other words, the software and processes must be in place to publish the early edition of the assets.

Agile Publishing

I've been invited to speak at George Washington University on the subject of Agile Publishing. This is a new application of Agile programming principles applied to content rather than to code. So I'm starting to explore the relationship and applicability.

The basic tenets of Agile Software Development (taken from http://www.agilemanifesto.org/) are:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

And Agile at work in software development usually means following these principles (http://en.wikipedia.org/wiki/Agile_software_development):

• Customer satisfaction by rapid, continuous delivery of useful software
• Working software is delivered frequently (weeks rather than months)
• Working software is the principal measure of progress
• Even late changes in requirements are welcomed
• Close, daily, cooperation between business people and developers
• Face-to-face conversation is the best form of communication
• Projects are built around motivated individuals, who should be trusted
• Continuous attention to technical excellence and good design
• Simplicity
• Self-organizing teams
• Regular adaptation to changing circumstances

So, how does this apply to publishing? Well, there are both philosophical and practical answers. Michael Fitzgerald published an article on XML.COM on this at http://www.xml.com/pub/a/2006/03/08/agile-publishing.html. In it, he examines the basic tenets of Agile in a publishing context and makes some specific suggestions on how to become agile. I'd like to extend his work. In the next several blog entries, I'll take a look at each of the principles in a publishing context. But first, let's look at what makes for a successful Agile project:

• The culture of the organization must be supportive of negotiation
• People must be trusted
• Fewer but more competent people are needed
  
• Organizations must live with the decisions developers make
• Organizations need to have an environment that facilitates rapid communication between team members

I believe this holds true for an Agile publishing effort as much as for an Agile software development effort. These key elements center around trust and communication. Find good people, empower them to success, trust them to get there and communicate regularly. It may mean compromising certain aspects of traditional publishing, such as corporate style oversight, high-overhead decision-making, and trusting decisions to workers.