Short-form fragments

Volume 14, Issue 5; 02 Feb 2011

I'm fairly obsessive about my data. Usually I'm content to stop at the point where I have a copy, but in the case of tweets, perhaps you should have a copy too.

I've been backing up my Twitter stream for a while. I think I have all my tweets, having gotten around to backing them up just before tweet #1 disappeared behind the limits of the Twitter API. My archive of the rest of the stream, the tweets of all the folks I follow, is spottier because the stream goes by faster.

Tim’s observations about the relationship between blogging and Twitter ring true to me. His follow up about injecting tweets into his weblog both sounded like a good idea and provided a small challenge: if Tim can inject his tweets into his weblog, can I do the same?

Well, yes, of course. I can wrangle XML to do just about anything. And so I have. The technical infrastructure that does the work is a MarkLogic server running an app not unlike the system I started to describe before. [You must revise and finish that someday, —ed.]

Like Tim, I'm going to publish them a week at a time on Mondays. Unlike Tim, I'm displaying the tweets (mine and the ones I mark as favorites) in the context of the conversations in which they appear. All of the tweets are displayed with their attribution. Tweets that I've marked as favorites are marked with a “star” icon (or the alt text “FAV” if you're not using a graphical browser).

There are all sorts of open questions in my mind about whether these generated essays should appear in feeds, on the homepage, etc. For the moment, I'm excluding all the archival essays and treating the essay for last week as a normal essay.

We'll see what happens…


Good to see you doing and thinking about this Norman, I've long been thinking about pulling my microblog posts, normal blog posts, and other things in to the same "personal data stream", however my purpose for this may differ to yours. I want to do it in order to analyse each tweet/post/message semantically (run it through silcc, open calais, zemanta etc) in order to see my interests and focuses over time, essentially it would index some of my "thoughts" and I could see where new things came in, when topics died (for me) and also have it suggest avenues of thought / topics that I've forgotten but are related. When you combine this with pulling your emails in to the same stream, your friends writings, and so on, it gets really interesting.

There are of course many other things that could be done with the data, but I guess my grounding theory, which may compliment yours, is that "my website" should be optimized for me, and if it, then it'll be more useful for other people.

—Posted by Nathan on 03 Feb 2011 @ 12:39 UTC #

You (or others; sounds like you're already set for tools :) might take a look at ... it can aggregate from Twitter, Facebook etc., stashes in SQL locally, and provides a Web UI (with plugins).

—Posted by Dan Brickley on 03 Feb 2011 @ 01:06 UTC #