Publishing and Mobile Devices

I've been interested for a long time in writing applications that will provide some useful service on a mobile phone, and with the advent of smart-phones I get to explore this a little more. This got me to thinking about apps that run on smartphones and that are possibly useful to information customers - apps that a publisher might develop and deploy to their subscribers. And since I have a new Droid phone, I started doing some programming for that. But it begs an interesting question - what phone(s) should a publisher target in developing a new app?

Of course, this is heavily informed by the particular publishing sector and by the kind of information being published. Phones, with a small-ish form factor, will favor small units of data; pads would more readily support larger. In fact, the form factor on a pad (like the iPad) is such that simply using a browser in the pad to access the same publisher web resource that one would access via a computer is usually an acceptable user experience. So, if I were a publisher, trying to decide what phone to develop an app for first, what factors might influence my decision? Here's what's floating around in my head:

1. Hard to ignore the overwhelming presence of the iPhone. Huge market penetration. But small form-factor. And it just plain pisses me off that I have to go through the iTunes marketplace and pay Apple a pound of flesh. It's my data after all! And I suspect most publishers are not really selling the app; they're selling content.
2. Hard also to ignore the skyrocketing increase in iPads. Seems like everyone has one these days! Wish I did. And, given that the iPad runs the Safari browser (which is Webkit compliant), I can develop a variation of my website using an HTML5/CSS3 approach, host it on my own servers and not have to go through iTunes/Apple. More on Sencha/jQueryMobile/PhoneGap in another post... But if I need to create something that runs on the iPad, I run into the same iTunes/Apple question.
3. Some of the stuff I'm reading indicates that the various Droid phones are increasing market share considerably. And I can develop an app for a Droid phone and distribute it myself without going through the Google market or, if I choose, I can go that route. 
4. It seems that in certain market segments Blackberry is ubiquitous. Especially in the legal and lobbying worlds. As with the Droids, I like the fact that I can deploy an App frmo my own sites and am not forced to go through a marketplace. The small form-factor makes the BB somewhat limited for larger units of data and the differences between versions of BBs and the attendant differences in API support make BB develop somewhat challenging, although a cross-platform approach such as HTML5/CSS3/PhoneGap may limit those difficulties. OTOH, RIM is trying to shift to QNX and allow Droid apps to run on Blackberries. But when? And how long will it take for that upgrade to penetrate the marketplace. And a lot of the stuff I read indicates that RIM is losing market share. 
5. And then there Microsoft. I sometimes think that if MS creates a phone that integrates seamlessly with their back-office servers (Exchange, Sharepoint, etc.), those would become the darlings of IT departments worldwide. Enough so that they could drive RIM out of that spot. But that hasn't happened and MS phones are distant cousins in terms of market share. 
6. Kindle Fire? I love the idea and the price, but it means limiting app development to Android 2.2. We'll see!

So, I think I still have more questions than answers. And I know that some of this is a religious argument - I hear fans of the droids espousing their virtues over the iPhones and vice-versa.

Semantic-Content Management

We've been developing a tool set, or framework, or whatever for several years. We call it "Tractare" which is Latin for "to handle, manage, perform". I'm a sucker for that kind of naming.

Anyway, what's it all about? Well, as the Internet becomes more saturated with raw information, keyword search engines really aren't enough to locate that needle in the haystack. We (content providers) need to describe our content in a way that users "get" -- we need to describe the "aboutness" of our content.

An example I like to use when speaking on this subject is this: If you were to use google to search for "retarded", you would get a gazillion hits. But few, if any of those hits would have come up using the politically correct phrase "intellectually challenged". This is because a keyword engine like google depends on the actual presence of the keyword, either as text in the content or as metadata. Now, you could encode both forms of this concept as meta-data on your web-page and it would be found. Now, if you are a psychologist or someone working in mental health, you'd probably be getting the results you want. But if you are a firefighter, the word "retarded" has a whole different meaning. How do we express that? The answer is in several parts of course. But first, we need to capture the meaning of the content; the "aboutness". We need to associate the "firefighter" concept with the content that pertains to fighting fires. This is what Tractare permits us to do.

Tracare is a framework. It's not an off-the-shelf product. It is built on the idea of topic maps -- organizing content around indexes and concepts. It's true power lies in a combination of searching and navigation tools that allow the user to narrow the scope of their work to a set of concepts. We build custom CMS and delivery solutions on top of it.

The CMS systems we build usually include features found in social networking, including folksonomies (as well as traditional taxonomy and classification support) and ranking/commenting. These features allow content providers to apply semantics to content in a number of new and different ways.

The delivery systems we build often include a number of search and navigation interfaces that web users have come to love, including mashups, classification searching, semantic browsing and so on.

Advertising-Based Publishing

In a previous post, I was exploring new trends in electronic publishing (April 30, 2008). One of the key points was that traditional subscription and publishing revenues are shifting away from subscription fees towards "free" information supported by advertising revenues.

"One well known revenue shift is towards advertising revenues. This is an old model, of course. Newspapers have been at it forever, and online search engines for years. But more recently, we're seeing a larger shift of revenue away from subscription sales for information resources and towards advertising."

I had this reinforced recently when I realized that even traditional reference book publishers such as dictionaries and encyclopedias were increasingly garnering revenue from advertising rather than from subscription sales. Wow -- that was an eye-opener. It makes sense though -- if I'm looking for an informative article on "Chocolate", am I going to pay for access to Britannica or would I go to the free Wikipedia? I guess the answer (for me at least) depends on how authoritative my answer needs to be. But in general, I would go to the free site. And I suspect I'm not alone. So how is a traditional reference publisher to compete in the age of Wiki-whatever? Product quality alone isn't enough. It has to be free too. Enter advertising.

On the surface of it, it seems easy to generate advertising revenues. Especially if your area of publishing is targeted - in fact, the more specialized your content, the more valuable you are to advertisers? I'm not sure that this is true, but it sure looks that way. Virtually anyone can establish a Google AdSense account and tie it into their content publishing operation (as I've done to this blog). But who's actually making money at this? That's the hard question.

It always comes back to the same basic principal - supply and demand. I loosely translate the supply side to "timely, quality content". Timely doesn't necessarily mean frequent, it means "frequent enough". And demand is partly driven by the content and partly by your sales/marketing operation. Demand is partly the number of visitors to your web site and partly by the number of visitors who pay attention to your advertising. So, we need quality content and we need to advertise its presence.

These sound like the principals as those by which we've been driven forever.

Gilbane, San Francisco, 2008

I spoke at the Gilbane Conference in San Francisco in June (http://gilbanesf.com/) on the subject of using folksonomies as a publishers tool for classifying content. The session moderator had warned me that attendance was a little thin and that we shouldn't expect too many in the audience, but my rough estimate was that we had about 45 people there. I was pleased. And there were lots of questions and a lot of discussion.

Here's the abstract from my presentation:

Folksonomies, Just Good Enough For All Kinds of Things
Extending Folksonomies to Describe All Kinds of Content

Folksonomies are gaining popularity in the content delivery world as a "good enough" way to classify content; tag clouds are becoming commonplace on content-driven website. More and more, folksonomies are working their way back into the content creation and management process. Huge volumes of content, increasing and diversifying user requirements are pushing content management operations to look to folksonomies as a Web-2.0 way to describe the "aboutness" their content.

With this simple concept in hand, why not use this technology to describe more than just content subjects? "Aboutness" is just one aspect of your content. What about order, threads, or related content? This talk will explore the extended use of folksonomies as a technology for enhancing content in new and different ways.

I can send the power point to anyone interested, please email me or leave a comment on this blog.

Using Folksonomies in Content

I've been thinking more and more about folksonomies as a replacement for traditional taxonomies. We can create all the tools we need, or work off of existing tools such as del.icio.us. But in the end, size does matter. For a folksonomy to work, a *lot* of people have to look at and tag the content. To me, this means content has to be exposed to readers, and lots of them, to tag the content.

Traditionally, the classification operation has been something done under control and part of back-room content management, by a small, select group of indexers. So first, we have to decide to relinquish some control. While this may seem scary, in reality the volume of tagging makes up for the lack of specific control. And, we can build the tagging tools so that editors and indexers can follow behind the tagging and clean it up. But if we can achieve a large volume of tagging, the volume and repetitive nature of the tagging will create common tags, threads and relationships.

Second, we need to find a large group of taggers. Depending on the nature of the content, this can be accomplished in a couple of ways. First, and perhaps easiest, is to expose content to the web. Perhaps through incentives, taggers can be enticed to tag. And, of course, staff of the publisher should be encouraged to participate as well. Failing that, a publisher could look at a human-automation engine such as Amazon's Mechanical Turk, where large numbers of minuscule tasks that are best done by people are spread out over many people for a small fee.

Folksonomies Applied

In a previous blog entry, I was starting to think about folksonomies as they might apply to content management. Having lots of experience in topical classification, therauri, and related tools, I figured this would be a simple discussion, but it isn't.

What is a Folksonomy? "Folksonomy (also known as collaborative tagging, social classification, social indexing, and social tagging) is the practice and method of collaboratively creating and managing tags to annotate and categorize content. In contrast to traditional subject indexing, metadata is not only generated by experts but also by creators and consumers of the content. Usually, freely chosen keywords are used instead of a controlled vocabulary" (Wikipedia).

By their nature, folksonomies are created by the people. I think the key to a successful folksonomy is participation by many people -- we make up for the lack of a controlled, standardized vocabulary and it's application to content by sheer volume and enthusiasm from a wide-spread using community. In fact, complaints about this approach usually center around the imprecise nature of the tagging. Since users typically apply tags to content, the tags are often ambiguous, overly personalised and inexact. But, Guy and Tonkin make a persuasive argument that user applied tags are in fact converging -- that the overall universe of applied tags is becoming self-limiting. If so, then the universe of tags that are being created in services like Flikr and Del.icio.us are becoming useful bases for classifications.

And in other, related developments, these services are beginning to categorize their tags. Especially at Del.icio.us where there are now classification tags and action tags, among others. To me these seem like we gray-beards call facets.

But that's not really my point, though I think it is important. I think we need some tools that can work with content management to allow tagging, maybe even super tagging, wherein the tags are members of controlled facets. This isn't really hard. Virtually every content management system "knows" about content by a URI. And there are some very cool features in the Del.icio.us service, including keeping my bookmarks and tags private, and retrieving them via an API later. So, we can set a bookmark in Del.icio.us containing the URI of the content we want to tag with the tags we want for that content.

I tried this very simply by registering at Del.icio.us, turning on the "private bookmarks" setting, and putting the Del.icio.us buttons on my browser toolbar. Then I pointed my browser at a content item in a CMS (Alfresco) and clicked the Del.icio.us "Tag" button. Added tags and saved it. I can see the tags and the URIs in my items on the Del.icio.us website.

But, to actually use this information, we need to pull the tags and content URIs back out of Del.icio.us. XML to the rescue. Or, rather, XML and the Del.icio.us API. We can fetch the tags we're using by using this URL: https://api.del.icio.us/v1/tags/get. We can also see all of our content using this URL: https://api.del.icio.us/v1/posts/get. And we can retrieve by tag: https://api.del.icio.us/v1/posts/get?tag=C

Pretty cool stuff.

Back to the GWU Summer Publishing Institute...

So, getting back to the GWU Summer Publishing Institute, and the topic on which I need to speak, "Agile vs. Traditional: Methods for Building a Software Infrastructure."

It seems to me that, after exploring what Agile Publishing might mean, it's clear that an Agile Publisher needs a software infrastructure that supports the key tenets. I take that to mean a content management and publishing infrastructure that supports the tenets of Agile Publishing:

• Customer satisfaction by rapid, continuous delivery of useful product
• Working product delivered frequently
• Working product is the principal measure of progress
• Even late changes in requirements are welcomed
• Close, daily, cooperation between business people and developers
• Face-to-face conversation is the best form of communication
• Projects are built around motivated individuals, who should be trusted
• Continuous attention to technical excellence and good design
• Simplicity
• Self-organizing teams
• Regular adaptation to changing circumstances

So what kind of framework is that? I'm envisioning a content management and publishing system that easily pushes out releases on a frequent basis, allows content and structural changes to be readily made, enhances communication between participants, enables remote meetings, provides templates for good design, has easy-to use process and product review tools, enhances workflow with loose rules, and can be easily adapted to changes.

This goes a ways beyond a content management system, or any publishing system on the market today because it demands flexibility and adds project collaboration tools.

In "The Emerging Art of Agile Publishing", XML.COM, 3/8/2006, Michael Fitzgerald makes some specific suggestions about how to be an Agile Publisher:

• "Build real trust through constant, informal communication
• Interleave work processes between writers, editors
• Share work openly, don't store it in secret silos
• Store source files in an online repository and give all members of the team access to it
• Agree on and share tools to reduce the waste of format conversions
• Carve up your work into small, easily consumed and exchanged pieces
• Keep track of what you are doing daily with some other suitable tool
• Publish your work early and often, charge a little something and get free reviews
• Let Herb do things his own way, but not on your team"

I would add to that, "Keep it simple".

So what software is needed? First and formost, a communications tool. Especially in these days of telecommuting, getting face-time is hard. I like instant messaging and the Skype product in particular. Skype has a secure chat and telephony in one kit. Second, a content management system that will permit easy and frequent releases of product. There are many of these, including several open source ones. Third, open project management. The basecamp.com approach seems nice as does the offering from project.net. I'm personally in favor of an open project document repository for sharing project related materials. This could also be used for openly managing project management materials, such as an MS project plan.

Agile Publishing -- Continuous attention to excellence, good design, and simplicity

Principles Number 8 and 9 of the Agile Software Development approach . What do these mean for an Agile Publisher?

As with some of the previous principles, this is partly just good business and I feel these are all closely related. Simplicity => Good Design => Excellence.

Especially when developing a complex product with many content components, it is easy to design an overly complex, difficult to build and maintain product. Simplicity is the name of the game here. Also, good design -- keeping sight of both the immediate need and the growth room and designing with these in mind.

Agile Publishing -- Projects are built around motivated individuals, who should be trusted

Number 7 of the Agile Software Development approach . What does this mean for an Agile Publisher?

It's all about trust. This also could be said of every project everywhere. There's nothing different here for an Agile publisher than for any other development effort. Michael Fitzgerald ("The Emerging Art of Agile Publishing", March 8, 2006, XML.COM) posits these tenets:

• People must be trusted
• Fewer but more competent people are needed
  
• Organizations must live with the decisions developers make

These are key points. Trust inspires dedication, dedication reduces numbers of people. But (I hear you cry), organizations must live with the decisions developers make? Hard to swallow, that. But remember, a key point of the Agile approach is that the development team includes stakeholders. It is partly their role to ensure that the decisions the developers make are in the best interests of the product and the client. A better product will come of it.

Agile Publishing -- Close, daily, cooperation between business people and developers and Face-to-face conversation is the best form of communication

Close, daily, cooperation between business people and developers and face-to-face conversation is the best form of communication -- Principles Number 5 and 6 of the Agile Software Development approach . What do these mean for an Agile Publisher?

It's all about communication. This could be said of every project everywhere. There's nothing different here for an Agile publisher than for any other development effort. Michael Fitzgerald ("The Emerging Art of Agile Publishing", March 8, 2006, XML.COM) posits these tenets:

• The culture of the organization must be supportive of negotiation
• Organizations need to have an environment that facilitates rapid communication between team members

Bravo.

Agile Publishing - Even late changes in requirements are welcomed

Even late changes in requirements are welcomed -- Principle Number 4 of the Agile Software Development approach . What does this mean for an Agile Publisher?

As the publishing world changes to include more and diverse assets in a publication, we introduce the possibility (or even probability) that changes will occur during the product development life-cycle. These could be as simple as changes in artwork or as complex as an overhaul in related assets.

One of the significant features of the iterative nature of Agile development is that the stakeholders (client reps) can see if the product is taking shape as was expected. If not, changes are not only possible, but encouraged, at just about any point in the development cycle. This same benefit obtains for changes to the underlying business model -- should the market conditions for the product change, we can respond to these changes.

Retaining this ability to accept changes to the requirements, no matter when they occur in the development cycle is a pre-eminant feature of Agile and applies equally to Agile publishing as it does to Agile software development.

Agile Publishing -- Working product is the principal measure of progress

Working software is the principal measure of progress -- Principle Number 3 of the Agile Software Development approach (http://en.wikipedia.org/wiki/Agile_software_development). Let's change this to "working product is the principal measure of progress" and explore what this might mean for an Agile Publisher.

Norbert Winklareth, one of the smartest people I know, asserts that the measure of value for which customers pay for is not any traditional metric of development but features:

1. Value => Functionality That Works
2. Functionality => Set of Features
3. Therefore: "Features are the true measure of development"

It would seem that the same principle can apply to a publishing project -- a publication has, among other things, a set of features. In software, features are specific characteristics, such as a search function or an RSS feed. In publishing, features could be harder to describe. In essence, these are the characteristics of a product for which customers pay. As a practical matter, features could be publication components (or assets) for example, written text, artwork, multi-media software functions, summaries or abstracts, and so on. Each of these can be considered as a feature of the published product.

Deciding on the priority of the features of our product is important here. By developing the most important features first, we can plan the creation of our product in stages (or iterations) in which we get our most desirable or most important features built first. The result is that we have a working, functional product at the end of each iteration that could be considered a final product if need be. It certainly permits us to respond to budgetary or other outside business pressures and still end up with a publishing product.

It also provides us with a true metric for progress. As features are developed, we can measure that easily against our plan for the product. We then know how well our project is fairing.

Agile Publishing -- Working Product Delivered Frequently

Working software is delivered frequently (weeks rather than months) -- Principle Number 2 of the Agile Software Development approach (http://en.wikipedia.org/wiki/Agile_software_development). What does this mean for an Agile Publisher?

One of the key issues that arises in traditional software development is that the client's vision of the features and functionality of a product are different from the developers' vision. Another is the inability of the stakeholders to control development against their budget. These two issues are at least partially addressed through the use of Agile's frequent delivery of working software principle. Specifically, by getting frequent releases, clients can see for themselves how conformant the developers' vision is to their own. Also, and especially when combined with another concept -- planning for priority features in the early releases, should the development cost begin to approach the project budget, the project can potentially be terminated early with much of the priority feature set in working order. Powerful stuff.

Developing a publishing product can follow a similar flow to software development, one in which iterations of content are created and reviewed. This is especially true for products with content derived from many sources. Following a similar flow of development, we can exert the same control -- create iterations of working versions of the product and with more important features (included content sources perhaps) developed first.

In the realm of electronic publishing, it is almost always true that a product is derived from many sources and I've seen many publishing efforts held up in order to include a particular content asset. I wonder if that asset was important enough to the product to incur the delayed release and resulting revenue.

Agile Publishing - Continuous Delivery of Working Product

Customer satisfaction by rapid, continuous delivery of useful software -- Principle Number 1 of the Agile Software Development approach (http://en.wikipedia.org/wiki/Agile_software_development). What does it mean for an Agile Publisher?

To my mind, this is one of the most important pieces of Agile. This is what ensures that what the customer envisions is what is being built and that if not, corrections/realignment can be done early and at the least cost. It has some secondary advantages as well: regularly tracking progress against a budget; early start on customer training; and easier collaboration with client/stakeholders, to name a few.

What does this mean to electronic publishers? Publishing describes a large, complex industry that delivers information to a user community. There are many forms of electronic publishing: newsletters, online books, references, electronic databases, and so on. There is a commonality to software development in that published products (or assets) go through a development/authoring process, a QA (or editing) process and delivery.

An Agile Publisher would apply the principle of "rapid, continuous delivery" to each electronic product. In practice this could mean author submissions of partial manuscripts, developer delivery of database subsets, and so forth. Any of these early and frequent data feeds provides a foundation for "rapid, continuous delivery", which the permits the goals of early and ongoing review and realignment, budget tracking, etc.

But, to take advantage of this, the electronic publisher must also have a deployment infrastructure that will create a working version of this electronic asset. In other words, the software and processes must be in place to publish the early edition of the assets.

Agile Publishing

I've been invited to speak at George Washington University on the subject of Agile Publishing. This is a new application of Agile programming principles applied to content rather than to code. So I'm starting to explore the relationship and applicability.

The basic tenets of Agile Software Development (taken from http://www.agilemanifesto.org/) are:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

And Agile at work in software development usually means following these principles (http://en.wikipedia.org/wiki/Agile_software_development):

• Customer satisfaction by rapid, continuous delivery of useful software
• Working software is delivered frequently (weeks rather than months)
• Working software is the principal measure of progress
• Even late changes in requirements are welcomed
• Close, daily, cooperation between business people and developers
• Face-to-face conversation is the best form of communication
• Projects are built around motivated individuals, who should be trusted
• Continuous attention to technical excellence and good design
• Simplicity
• Self-organizing teams
• Regular adaptation to changing circumstances

So, how does this apply to publishing? Well, there are both philosophical and practical answers. Michael Fitzgerald published an article on XML.COM on this at http://www.xml.com/pub/a/2006/03/08/agile-publishing.html. In it, he examines the basic tenets of Agile in a publishing context and makes some specific suggestions on how to become agile. I'd like to extend his work. In the next several blog entries, I'll take a look at each of the principles in a publishing context. But first, let's look at what makes for a successful Agile project:

• The culture of the organization must be supportive of negotiation
• People must be trusted
• Fewer but more competent people are needed
  
• Organizations must live with the decisions developers make
• Organizations need to have an environment that facilitates rapid communication between team members

I believe this holds true for an Agile publishing effort as much as for an Agile software development effort. These key elements center around trust and communication. Find good people, empower them to success, trust them to get there and communicate regularly. It may mean compromising certain aspects of traditional publishing, such as corporate style oversight, high-overhead decision-making, and trusting decisions to workers.