It was my second year at FOWA UK and although I left with a feeling that I saw nothing new or innovative I did leave feeling inspired which to me is what this conference is all about. Ryan Carson and the Carsonified team definitely deliver the best conference in this space and the fact that there will be hundreds of blog articles by people reflecting on the conference this week has got be a great complement.
The conference sessions were split into two tracks - “developer” and “business”; obviously you can’t see everything but all sessions were being filmed in HD and video, audio, slides and even transcripts were being uploaded to http://events.carsonified.com/fowa/2008/london/content on the same day - if only every conference did this!
To me it felt as though there is a shift in the air and that web 2.0 is in a state of transition accelerated by the global financial conditions. The speakers and tone of the conference in the main were responsive to this shift which was especially noticeable on the second day.
I must admit that I probably did "waste the entire time attending talks and panels" but I have come away with thoughts and notes that I’ll be blogging as 4 entries; 2 days am and pm - so, in terms of personal reflection probably not a waste of time.
Day 1 morning...
Personal highlights:
- treating friends as an information filter
- measuring relevance and reputation of friends
- catering for localisation
- designing for “scaling out”
- plan for N>1
Kevin Rose - The Future of News
First up was Kevin Rose from Digg talking about how Digg currently is a “zeitgeist of users at that time” and how they want to move on and create an experience that you will enjoy by “spreading” the long tail information and getting “diggs” to the people that it may be of interest to - essentially personalising the digg experience to get beyond the collective intelligence / “madness of the mobs” homepage and to use the social graph to benefit you and to be of benefit to others.
This introduced the concept of “friends” as an information filter that was repeated many times throughout the conference by various speakers.
Digg currently operates in two spaces (rather like two domains in communities of practice terms) - people you know and the unfiltered masses. There is a space in between here where similar users to you can be automatically identified and their diggs and buries shared with you. Digg’s implementation of a “recommendation engine” has increased friends activity by 4x and overall increase in diggs by 40% since it was introduced early in the Summer proving that there is something in this “friends as an information filter” that tangibly increases interest and traffic.
Other improvements on the roadmap for the next year (Series C funding secured recently to see this through) included opening up their taxonomy to encourage niche content; promoting conversations between users; knowing the “impact” of your diggs and buries and an increase in publishing tools / API to further digg as a conduit to push content to other social networks.
Intriguingly a scenario where the BBC could expose other BBC articles of relevance upon a digg if the BBC adopted the digg API was illustrated.
Lots of questions were put forward with quick & sharp answers ranging from introducing reputation via segregating recommendation engine buckets where a user may be recognised in one area but not necessarily in another; location based services and tailoring and weighting content based on location; revenue streams and monetisation essentially still advertising and primarily the 3 year Microsoft deal and localisation where Kevin stated that this would be a big initiative in 2009 and by Q4 would be moving servers overseas to cater for this.
Overall a very nimble keynote introduction to FOWA primarily focusing on Digg but the questions drew out issues relating to all web apps with technical / infrastructure related questions being passed onto Joe Stump who would follow later in the morning.
To me it felt as though Digg is looking to step beyond what Newsvine has done by and to become more embedded and as a conduit for large content creators whilst retaining but possibly segregating its current community.
Edwin Aoiki - Web Apps are Dead, Long Live Web Apps
Edwin from AOL drew a comparison with web 2.0 and the current global financial crisis (GFC) in the context of unsustainable products and solutions where people are drawn in to “shiny new things” and don’t care about the underpinnings (standards / financial instruments) until things go wrong...
Edwin focused on creating enduring value beyond monetisation models and page impression type advertising to creating communities that have pride in and can “contextualise” your product or solution - “it’s not just about the money”. These communities are not just limited to external customers but also include the internal team and the external developer networks.
Edwin ended by introducing the concept of the liquidity of ideas as an ecosystem of sharing that can be built into your business and offering that can potentially provide other means of sustainability and future monetisation as well as tactical information for which direction your application and its community wants to and perhaps needs to take.
Blaine Cook & Joe Stump - Languages Don't Scale
Blaine Cook (Twitter, Odeo, & Yahoo! amongst others) and Joe Stump (Digg) delivered a fantastic double act about scaling with beautiful background images of babies and kids in their slides. Overall this was a great retort to the language zealots that have given Blaine a hard time earlier in the year re: twitter and the fail whale.
Blaine and Joe quickly launched into a theme of PHP sucks!, Ruby sucks! and Python sucks! with short reasons from their viewpoint as to why. Results from a quick and “non-scientific” experiment were shown that illustrated the negligible CPU difference between PHP and Ruby where a “fetch and parse RSS feed” task was repeated 100 times (a very common process) - the results highlighted simply how web apps do not and should saturate the CPU and instead the scaling issues are all about the IO - the disks and network.
Scalability and “Scaling up” and “Scaling out” were differentiated as scaling up via box to bigger box to balanced cluster to larger balanced cluster etc. and scaling out as RAIC (Random Array of Inexpensive Computers) where “cheaper & crappier” boxes can be thrown at the problem!
Continuing the “sucks” theme “Web 2.0 sucks!” in that the community can create a lot of “crap” that requires architecture and engineering to manage successfully. The approach put forward was to encourage “happy coders” who have the tools and environments that they want to use rather than forcing a language or architecture upon them.
Blaine and Joe then addressed a number of ways to improve IO including “caching everything” (memcache for Digg approx 1Tb; Flickr 3Tb & Facebook 7Tb); expiration jittering to ensure all caches do not expire at the same time; use of queues and messaging handled by a non web request machine(s) and horizontal & vertical partitioning of data.
Questions from the floor drew out the common problem of the walled garden between developers and operations, the consensus here appeared to be that the best developers had either carried out an operations role or served time there first. Capacity planning and forecasting disk and network requirements by accurate monitoring of metrics and trends was also a good discussion point. On the “scaling out” front the area of data backup of these cheap redundant servers highlighted the need for an designed architecture that accommodates multi-home and failover and the subject of another talk (and presumably what Joe and Digg are going through now to plan for their international growth).
Matt Biddolph - Made of Messages
Matt from Dopplr continued the developer track with his session of message queues. Matt kicked off with the Unix & command and a nice analogy about a sense of “gearing” where “small pieces loosely joined” can be processed in the background in order to shift away from a sequential approach - does it matter if someone else's updates and activities are not simultaneously and immediately updated and shown in your timeline? The whole approach of message queues and distributed processing is not new and large corporate closed systems, in particular the banking sector, have been doing it for years - and all this web 2.0 community “crap” has resulted in a need for queuing systems.
N>1 was introduced highlighting the pain required to go from “1” to “2”, from one DB server to 2 DB servers; 2 DB servers to a DB server farm etc. Matt’s advice was to prepare and plan for this N>1 so that you can rapidly implement in a controlled manner should you need to rather than scrambling around trying to satisfy growth demand.
Metrics and drawing out information from those metrics is essential to understand the “health” of your app and its environment and Matt introduced distributed message queues as a great “free” way of providing this information that can be pulled out from the live flow of data flow at will.
Overall a decent start to the morning in the developer track introducing the social graph and catering for growth for when the activity and content produced by this community increases.