Adventures in home working

My current Adult Onset Stills Disease flare seem to be nearly over now and I should be back at work in a couple of weeks.  This is my longest flare, lasting from Mid January to the end of May.  Its not been too severe because now I am diagnosed the drug treatments keep me fairly stable, but its not very comfortable and its difficult to work through the pain, fatigue and difficulty concentrating. 

For those of you who are interested, probably only fellow AOSD sufferers I have attached my ‘end of flare report’

I came across across a medical paper on AOSD the other day, but it was so full of jargon I couldn’t follow it.  I created an annotated version which I have posted here in case it’s of any use.  Not being a medical person myself don’t take it too seriously and if you have any questions about the content please check with your specialist as I may have got some of the definitions wrong!

This site has just appeared http://www.reallysimplesyndication.com it includes the following bullet list of things that make up RSS.

RSS is…

1. A format.

2. Content management tools that generate feeds in the format.

3. Aggregators and readers that subscribe to the feeds.

4. Search engines and utilities that crunch the information and ideas.

5. Services from technology companies like Microsoft and Apple.

6. Authoritative publications like the BBC, The New York Times, CNET, InfoWorld, PC World, Time, Wired, Salon, Yahoo, Reuters — that distribute news and opinion in RSS.

7. Many thousands of weblogs covering virtually every aspect of life on this planet.

8. A vast and growing community of thinkers, writers, educators, public servants, and technologists.

The revolution of RSS is what people are doing with it, what it enables, the way it works for people who use technology, the freedom it offers, and the way it makes timely information, that used to be expensive and for the select-few so inexpensive and broadly available.

RSS is the next thing in Internet and knowledge management. It’s big. A lot bigger than a format.

In this blog I talked about a generic concept of operations associated with a conceptual information lifecycle. 

however with the advent of RSS, we now have an Open and Simple way for applications to publish, for users to locate and  subscribe and for subscribed content to be accessed, processed and ultimately scanned and consumed, discussed, archived and subsequently retrieved

In this article talk about my personal application of tools and techniques to that lifecycle.

1. Publishing, I used to use radio for all of my web publishing, either directly through the Radio Userland Web UI, or through MailEdit which provides an email interface which I can use through my BlackBerry.  MailEdit uses a number of directives to define for example the title for the entry and the categories that it belongs to.  I use autotext on the BlackBerry to make entering these easier.  I now use Blogmedia that uses the BlogWare SW.  I tend to write the entries in Word though because the screen area is bigger and because of the spell checking, and then just paste and post.  You can read my blog here
2. Location, like most people I locate RSS feeds in a wide variety of ways.  The primary way is through other people’s recommendations, but I also subscribe to some key magazines like eWeek and InfoWorld.  You can find a list of the RSS feeds that I subscribe to here.  The list in contained in an OPML file.  If you download it you will be able to import it into most RSS aggregators.  In addition I have a few searches that return their results as RSS feeds.  I personally use Feedster as my RSS search engine.
3. Subscription, I subscribe to all of my RSS feeds, Atom feeds and NNTP feeds using NewsGator which is a plug-in for Outlook 2003.  I really like the Outlook environment and use it for my personal email so it seemed a natural choice for me but there are many other options.  I will talk later about why NewsGator is such a powerful tool.  I use the NewsGator online service as well because it allows me to access my feeds from the web and synchronize all of my machines and keep a single copy of my subscriptions.  Subscribing in NewsGator is as simple is right clicking on any feed in IE.
4. Access and processing, all of my feeds are delivered into a separate local outlook PST file, i.e. not my mail file, each feed has its own folder which is automatically created.  For feeds that just have a link to a web site, i.e. they don’t include enough information to assess whether you want to read the item I use a custom style sheet which grabs the web page and saves that instead of the RSS description. 
5. Scanning, I have far to many subscriptions to read them all so its key that I am able to scan through them all really quickly.  This is one of the key strengths of Outlook 2003.  I simply create a search folder, which shows only unread entries from every folder under my root folder.  By doing this I get a single list, (grouped by folder name), which I can very quickly scan.  I also create a button on my toolbar that sets all entries in the search folder read, I click this each time I complete a scan.  As I scan I do one of two things, I flag an entry for follow up, or I open it in my browser.  Once that’s done I am ready to start reading.
6. Consuming/reading, OK so now I have a few flagged items in outlook which I want to come back to later, so I ignore these, they are easy to find because I have a search folder that aggregates all flagged items.  I also have maybe 20-30 articles that I want to read in more detail, write a blog entry about, forward to someone etc.  How do I manage all of those open browser windows, I use MyIE2.  MyIE2 is a wrapper for IE that implements a tabbed UI, I find this fantastic because it allows me to open as many windows as I want and then come back to them for some serious reading.  In addition as I read I often want to open more windows and I can do that in the background with a middle mouse button click on the link.  Once all of my background windows have opened I can go and review their tabs.  I can delete windows as I read them with a double click on each tab.  It gets a bit better than this though, because I have two monitors I have Outlook on may main monitor when I am scanning, MyIE2 sits out of the way but visible on my second monitor.  Once I start reading I switch them over.  One other great little MyIE2 feature is that if I have 20 web pages waiting to read and I want to restart my PC, no problem, I just save them as a group and reopen the whole group of pages later.
7. Discussion, I don’t tend to have enough time to do a lot of commenting on other people’s blog entries, but if I do I do it while reading from the web page for that entry.
8. Archive, all of my blogs sit in a PST file, which is fairly scalable, however I auto-archive items that are a few months old to a separate PST.  This makes no difference to retrieval because the tool I use searches both.  In addition I often use Google to search online, but I find it easier to find things I have seen and want to find again by reviewing my personal archive.
9. Retrieval, I have tried lots of local search tools but in the end I settled on X1.  X1 indexes all of my outlook feeds, emails, contacts, and files, (and promises to do Notes soon).  It has some great features.  As you type it shows you the matching documents in real time, its easy to filter the results based on full text, subject, author, folder, date etc.  It has a great preview window, which is particularly good for files.  X1 just sits on my second monitor waiting to jump into action.

My first thought when I cam across IBF, (who could miss it!), was that it was another Microsoft thick client solution.  I am still not sure but it looks like it might be a bit more creative than that for the following reasons:

1. I have always been a big advocate of standardizing the infrastructure capability layer and integrating it with the line of business layer.  That way an enterprise has its infrastructure in common, regardless of which process or division of the company you work in.  IBF looks like it addresses that need pretty well
2. I have also felt that the ad-hoc processes and information and collaborative processes are under emphasized in businesses that have a lot of formal mega processes that they like to optimize.  I talked about this in another blog entry.  IBF allows you to integrate ad-hoc and formal business processes.
3. It seems that a lot of thought has gone into making the maintenance of the IBF client environment as ‘thin’ as possible.  It still requires a client component to get started though.  I need to look into this more to be sure but it looks hopeful.
4. Its all about consuming web services, caching them at the client, and rendering them in a standard way.  This seems to be to be a major part of what WinFS, and the Longhorn Shell are all about so its good to see some early thinking in this area.  I have talked about this a little here and here
5. Although its being hyped Microsoft seem to be doing a good job of positioning it appropriately
6. In a way its a bit like gmail,  parsing email content for example and providing link to web services that might be appropriate, (in gmails case adverts but its easy to see the similarity), so it will be interesting to see how server side equivalents of the IBF idea take off now that Microsoft have given people the idea.

PC World talks a bit about Web Services for identity management.  The list of vendors demonstrating interoperability, (Netegrity, Oblix, RSA Security, OpenNetwork Technologies, and Ping Identity), as well as MS and IBM is pretty impressive. 

Web Services seems to be one of the few areas where MS seems to be getting its story straight, if my impression is correct:

1.     Very involved in the standards process

2.     Perhaps to the point of dominance of the standard process?

3.     Building great tools and middleware in VS and Indigo, so that their platform is the best way to implement the standards

4.     Building an innovative client environment for the caching and presentation of web services, WinFS and Avalon/WinFS Shell

5.     Integrating Office and Collaboration processes with line of business processes, Information Bridge and InfoPath

That’s not to say that there are no problems in these products.  But the story looks fairly coherent at least.  Have a look at this MS document if you want more details.

Infoworld talks to David Thompson, a Microsoft corporate vice president who has been in charge of the Exchange Server group since early this year.   It maybe just me but I get the distinct impression that the Exchange Group is in a bit of a state.  They don’t seem to really know where storage is going because of the flux around Longhorn server, and they don’t seem to know where to take Public Folders and other Document management like capabilities because they are dropping the old store and because of ‘competition’ from SharePoint services.  Core email does not have much growth potential in it, so all that seems to be left to work on is Edge Servers!  Not a very exiting roadmap!!

Last year I thought that the ‘big thinking’ that must be going on in Microsoft was starting to pay off in terms of well architected products that did not overlap and were being rebuilt from the ground up.  Now I am not so sure,  I see the delays in the foundation layers, ie SQL Server and Longhorn Server, triggering panic in the Office tools and Collaboration space, who are probably starting to think more short term about customer driven next versions, rather than strategic re-architecting of products.

Chris Sherman at SearchEngineWatch talks a bit about Longhorn search and links to a Channel 9 video clip where you get some glimpses of how search is going to be implemented in Longhorn.  What’s clear right now is that they are:

1.     Planning for a search experience that operates at multiple levels, local, intranet, internet etc

2.     That at the local level integration and meta data is going to be key.  The example sited on the video goes something like this:  “find me all of the PowerPoint presentations, attached to meeting invites that I attended in the last month attended by someone from Shell”. This links together a lot of WinFS metadata.

3.     Interestingly almost every WinFS search scenario is based on integration of other things with Outlook, for example, “Where I was at a particular time”, information or contact information or email context.

4.     The next version of Outlook and its integration with WinFS and some sort of super journal in WinFS are going to be key

In my case, for now at least, I have settled on X1 for the following reasons:

1. I like to search in scopes, ie emails, files
2. The type down experience is great
3. The ability to filter based on the content of fields is ace
4. The preview window, Inside Out viewers, at the killer feature for me
5. They are planning to support Lotus Notes in the next version

The Problem

Internet users have largely given up trying to keep up to date with the vast amount of content being published on their ‘favourite’ web sites, let alone the slow moving sites that they need to track but are not motivated to visit ‘just in case’.  Portal vendors have tried to help by allowing users to aggregate bits of many web sites together, to minimise the number of web sites a person needs to access, particularly in a process context.

Proprietary approaches to Syndication, or the publish and subscribe model to information access has been tried several times on the internet, taking the form of for example Internet Explorer channels, and PointCast personalised news feeds.  Avantgo continues to find a niche publishing channels to PDA’s.

Email has become flooded with newsletters, status updates, just in case cc emails, and application specific notifications.

RSS to the rescue

Recently however with the advent of RSS, we now have an Open and Simple way for applications to publish, for users to locate and  subscribe and for subscribed content to be accessed, processed and ultimately scanned and consumed, discussed, archived and subsequently retrieved.

Background

For information on RSS, have a look at this web site:

http://radio.userland.com/allAboutRSS

for its history etc you are best referred to Dave Winders history of RSS at:

http://blogs.law.harvard.edu/tech/rssVersionHistory

Given the simplicity of RSS, (most people assume RSS stands for Really Simple Syndication), many people overlook its huge potential, which arises not from RSS itself, but from the concept of operation that it enables, (although popularity must in part be attributed to its simplicity), and the passionate way in which its users promote that concept, mainly right now through their own blogs and their new found productivity in subscribing to internet content change.

This short article attempts to describe why I think RSS, and its associated Concept of Operations is important.  So lets start with what I mean by the Concept of Operations.  I mean a set of conceptual ways of doing things that are enabled by this bit of standard technical glue we call RSS.  There are probably many more concepts than the ones I am going to describe here but the list of keywords in bold above is enough to be going on with.  The key thing to note is that before RSS there were largely only application specific ways of achieving most of these conceptual activities.  RSS has enabled an interoperable standards based environment where hundreds of different applications work together throughout the information lifecycle.  The result is a rapid evolution of applications, built on a stable standard, (although there is some fragmentation in RSS versions and the competing ATOM syndication specification this has little practical impact for end users).

Concept of Operations

To help to show why I think RSS is important I have used the following conceptual information lifecycle, (although I have ignored the process of information creation in this article).  The lifecycle I have used starts with ‘applications‘ that need to publish data, which users then need to locate and if they are interested in, subscribe to and for that subscribed content to be accessed, processed and ultimately scanned and consumed, discussed, archived and subsequently retrieved.

When RSS is applied to these conceptual activities, a particular Concept of Operation is enabled that is particularly effective.  I will describe each part of the lifecycle and attempt to describe where RSS fits in and how its helps.  In a separate article I will describe how I have selected a particular set of applications and how there have been applied to the great benefit of my own personal productivity.

Applications

Ok so most people think of RSS as a way of publishing news, blogs, or what’s new information from web sites.  But the scope of RSS can be much broader, I find it best to think of it as a means of notification.  Many applications need to allow people to be notified about things, and RSS is the way to do it.  Microsoft employees have been evangelising RSS for a while, but recently Bill Gates down have started to talk up the technology which is important if it is to break into mainstream IT.  If you think of RSS in terms of some familiar Microsoft products you can image RSS being used to notify people about changes to Exchange Public Folder contents, SharePoint Lists including Document Libraries, Event log contents, Content Management Server web sites, SharePoint approvals etc. 

To use some examples from Industry you can image RSS feeds for service bulletins describing problems with cars subscribed to by garages, change notification for important documents like aircraft maintenance manuals, tracking of documents posted to knowledge management tools, publishing new customers or key events that happen in a CRM system etc.

Seen this way you can see why I prefer to think of RSS as a way for applications to publish change.  Using the word change can also be misleading because in some cases the published content may be largely static, that’s no problem to RSS, the benefit is that if it does change then the consumer will be notified.

Publish

RSS defines a simple XML format for publishing information.  The XML format defines a channel or feed, which is made up of a number of items.  Each item can contain a small number of elements including a Title and a Description.  Multi-media content can be included in enclosures and application specific attributes can be added as well.  The full definition of the RRS 2.0 XML definition can be found at:

http://blogs.law.harvard.edu/tech/rss.

So RSS at its simplest from a publishing standpoint provides a very simple way to provide a list of things that have changed, each RSS channel has a build date which ensures the consumers of the channel don’t download the feed unless it has changed and each item has a GUID, (unique id), that ensures the consumers of the channel don’t display items multiple times.  In collaborative scenarios an RSS feed can include a URL to a location where comments on an item can be recorded.

In many ways RSS is so trivial that its difficult to see what all of the fuss is about, the reason is simple and powerful, RSS is a standard, it allows for many different applications to publish in a standard way, and that means that the REST OF THE LIFECYCLE, i.e. the remaining parts of my little Concept of Operations can also be standardised regardless of the application!  That’s the power and prior to RSS the only thing to compete with RSS in its ubiquity was email and email got to be so successful for the very same reasons the simplicity and standardisation of SMTP.  If you want to understand a bit more about how email and RSS compare check out this link:

 http://weblogs.asp.net/alexbarn/archive/2004/05/22/139461.aspx

and this link:

http://www.windley.com/2004/03/04.html#a1072

Location

There are probably 6 main scenarios that involve locating information published via RSS that I can think of:

1.      You locate an information source while browsing the web, you see a little orange XML image that links to an RSS feed that you can subsequently subscribe to.  This is the most common experience today, i.e. you find something interesting and you subscribe to it so that if anything new arrives that’s interesting from the same source you will automatically be notified without having to go visit that web page ‘just in case’.

2.      Someone sends you a link via email, or in a document etc

3.      A web site or application presents an RSS feed to you as a value added service, for example an application vendor may provide a feed that contains information about updates to their software, or a holiday company that provides information about special offers you may be interested in.

4.      You use one of the growing number of search engines that regularly search thousands of RSS feeds for you, for the search terms you specify, and return the list of matches as an RSS feed

5.      The above generalised to search other content in addition to RSS feeds and return the results to you as a RSS feed.

6.      You subscribe to a feed because of a specific business scenario and in a corporate environment the subscription may be automated and part of some formal role definition you have, in the same way that you might have some corporate standard web browser favourites configured for you.

By this time you are probably thinking that most of these scenarios are currently served by email and you would be correct, the key question is how well are they served, and how extensible and scalable is email by comparison with RSS, that’s what I hope to answer as this article progresses.

Subscription

At this point things start to get interesting because the standards bit of the process has nearly come to an end and the innovation can start.  There are many applications serving many diverse needs that allow you to subscribe to either a single RSS feed or many feeds normally using some form of aggregator.  Single feeds can be displayed as part of web portals, emailed to you, downloaded and reviewed off line on almost all client platform types and operating systems.  Most people however use either a web based aggregator or a offline aggregator that runs on the client device, these can either be standalone or integrated with your email client, or even your web browser.  The key thing about most aggregators though is that they automatically and periodically pole often hundreds of applications, (mostly web sites), to check for changes and download those changes so that they can be processed on mass at your convenience.

Its also worth noting that RSS 2 also supports the concept of subscribing to PUSH notifications that a channel has changed.  By convention Subscriptions to Notifications need to be renewed every 25 hours.  Notifications take the form of XML-RPC or SOAP messages from the publisher to the subscriber.  The action taking by the subscriber on receipt of the message is not defined but it would usually be to read the associated channel.

Access

This is the easy bit, get the XML file using http.

Processing

Though its not necessary most RSS clients then perform some form of processing.  For example applying a style sheet or other transform to the XML to create a HTML representation.  In addition some will aggregate multiple channels and items into news pages.  There is much variety and innovation at the client end.  In a separate post, following this one, I will describe how my own environment is setup.  If you want client software, then I suggest you check out one of the following links:

http://www.ourpla.net/cgi-bin/pikie.cgi?RssReaders

Scanning and consuming/reading

Most people don’t read all of the items they subscribe to they scan them, flag in some way the ones of interest and then read them.  They then either delete them or archive them for subsequent retrieval.  Of course I used the word consume, rather than read because in the general case there is no reason why it has to be a person reading the feed.  If RSS is a general purpose notification mechanism then there may be automated consumers of those notifications.

Discussion

Most RSS hosting systems provide the ability to comment on an item, and for commenting on comments.  Authors can be notified when comments are made on their items.  This provides for a simple collaborative environment, although in many cases the client side support for threaded discussion type comments is not as efficient as one would expect based on the maturity of NNTP news readers that provide similar capabilities.

Retrieval

RSS items are increasingly seen as being a valuable information resource in their own right, as are the associated comments.  A person’s own individual archive of feeds is often very valuable for the ‘I know I read about this somewhere’ type of query and web RSS search engines are valuable for broader queries, especially for getting views of breaking news of any type.  For more information on search engines for news, including RSS see here:

http://searchenginewatch.com/links/article.php/2156261

Just finished my first read of this book, (it really needs reading twice), and found it pretty good.  Unfortunately it also left me being very frustrated because the methodology proposed is so well supported by the outlook Add-in, and I have to use Lotus Notes.  There is a template for Notes, but so far as I can figure it only seems to be available through coaching sessions.

There are some very good blogs on how to refine the methodolgy:

http://blogs.officezealot.com/marc/archives/000582.html

http://www.shahine.com/omar/PermaLink,guid,a8bb7588-179d-4d6f-aa5e-703ea2275e57.aspx

and of course Dave’s blog

http://david.davidco.com/

One of the key concepts in the book is, “A mind like water”, ie emptying the mind so that it can focus on what needs to be done.  The concept is sometimes misunderstood and there is a great post here that explains it very well.