Search Solutions 2010: Topics & Titles

Quick update to follow on from my previous post – we now have titles for all the talks at Search Solutions 2010. Next up is the panel topic, which I’ll post out about shortly, and the full programme, which we’ll publicize once we’ve nailed down the precise running order. Official blurb as follows:

SEARCH SOLUTIONS 2010

Thursday 21st October
BCS London HQ, Covent Garden

Search Solutions is a special one-day event dedicated to the latest innovations in web & enterprise search. In contrast to other major industry events, Search Solutions aims to be highly interactive and collegial, with attendance limited to 60 delegates. The high quality technical programme includes presentations, panels and case studies in which delegates present and share opinions, insight and expertise. The event should appeal to researchers, practitioners, consultants, end users or indeed anyone interested in the latest developments in web and enterprise search.

We will have talks from the following speakers:

Behshad Behzadi, Google: “Web Search  Freshness”
Vishwa Vinay, Microsoft: “Click evidence – Signals and Tasks”
Greg Lindahl, Blekko: “Instant Indexing”

Nick Patience, 451 Group: “The trends shaping the future of enterprise search 2010-2013”
Dusan Rnic, Endeca: “Enterprise Search and its Evolution”
Chirag Gandhi, mPhasis: “I Still haven’t found what I am looking for…”

Charlie Hull, Flax: “What’s the story with open source? — Searching and monitoring news media with open-source technology”
Till Kinstler, German Common Library Network: “Current trends in library search: From electronic card boxes to large scale, aggregated search engines”
Roberto Cornacchia, Spinque: “Search by strategy”

Mihai Lupu, Information Retrieval Facility: “Scaling up innovation”
Rob Stacey, TrueKnowledge: “Reconciling facts: how to check the consistency of facts created from web crawling”

Provisional programme:

09:30 – 10:00 Registration and coffee
10:00 – 13:00 Introduction, Session 1 & 2 + Coffee break
13:00 – 14:15 Lunch
14:15 – 17:20 Session 3 & 4 + Tea break + Panel
17:20 – 19:00 Drinks reception

The event will also be followed by the IRSG AGM (timing TBD).

Search Solutions is organised by the Information Retrieval Specialist Group of the BCS (British Computer Society), and is held at the BCS Central London Office:

BCS, 1st Floor
The Davidson Building
5 Southampton Street
London, WC2E 7HA
http://www.bcs.org/server.php?show=nav.8404

Note that there is no on-site registration.
*** Early Bird Registration ends: Thursday 7th of October ***

For further details, including registration, see
http://irsg.bcs.org/SearchSolutions/2010/sse2010.php

Read Full Post »

ECIR 2010 – Industry Day

Not sure if I mentioned this before (I probably did in the last issue of Informer), but on April 1st next year Mike Taylor, Udo Kruschwitz and I will be organising an Industry Day at ECIR 2010. This event is similar in spirit to the Industry Track at SIGIR (which this year was chaired by my ex-Endeca colleague Daniel Tunkelang), but with a more European focus. The venue is the Open University campus in Milton Keynes, UK.

In common with previous IRSG Industry Days, we aim to present a topical selection of talks, panels and case studies from the world of search and information access, with the emphasis on content drawn from the IR practitioner community.

We won’t be issuing a “call for papers” as such, as this is essentially a ‘by-invitation’ event. However, if there are individuals or organisations out there who’d like to get involved (e.g. as panellists or speakers) and have an interesting case study or demo to present, then drop me a note either here or offline.

As in previous years, we aim to present a topical selection of the latest innovations in search and information access, whilst maintaining the collegiate spirit of what we like to think of as a community event. (As an example have a look at last year’s programme: http://irsg.bcs.org/SearchSolutions/2008/sse2008.php ).

We won’t be issuing a “call for papers” as such, but if there are folks out there who’d like to get involved (e.g. as panellists or speakers) and have an interesting case study or demo to present, then drop us a line at the address below.

Read Full Post »

New Informer – Autumn 2009

We’ve just published the Autumn 2009 edition of Informer, which is the quarterly newsletter of the BCS IRSG. This edition features the following articles:

• Editorial by Tony Russell-Rose
• Conference Review: Search Solutions 2009 by Keely Flint
• Feature Article: UI Design Patterns for Search & Information Discovery by Tony Russell-Rose
• Book Reviews:
  • “A Generative Theory of Relevance” reviewed by Jianhan Zhu
    
  • “Introduction to Video Search Engines” reviewed by Nilesh Thatte
    
• Events listing edited by Andy MacFarlane
• Industry News, Featured Jobs, Contacts, etc.

 

Informer is free to download from the IRSG website. Enjoy!

Read Full Post »

Search Solutions 2009: highlights and reflections

Last Thursday I had the pleasure of co-organising & presenting at Search Solutions 2009, at BCS HQ in Covent Garden. Here are some of my highlights & reflections on the day.

These days there area great many events to choose from if you want to find out about the latest innovations in search and information access. But what makes Search Solutions different is that it provides an informal, interactive alternative to the highly choreographed, glossy brochure, convention hall style of conference that seems to dominate the search industry. Search Solutions is a much smaller, more intimate event, structured so that it is possible to meet just about all the speakers and delegates during the course of the day. In addition, the focus is firmly on the presentation of original material with credible technical content, rather than ‘sales pitches’ that contain no element of self-critique or realistic objective evaluation.

BCS President Alan Pollard

In common with previous years, this year’s event attracted an eclectic mix of speakers drawn from global organisations, start ups and solo players in the world of web & enterprise search. We were privileged to have BCS President Alan Pollard formally open the proceedings with an overview of the Society’s ambitious plans to reposition itself as the “Chartered Institute for IT”, providing an overarching certification body for all the specialist professions within the field. The technical programme was kicked off by Mike Taylor, who presented a overview of some of the techniques that have been used for learning ranking functions at Microsoft Research. It had been a while since I’d seen anyone provide serious mathematical analysis in a ppt presentation, but Mike more than made up for this, giving us enough equations in 20 mins to last a fair few conferences to come.

This was followed by Vivian Lin Dufour, who presented details of Yahoo’s progress in developing an interactive search assistant known as Search Pad. The idea of this app is to detect when users are engaging in research activity and support them with automated tracking, authoring and annotation tools. I think it’s a great idea, although not entirely unprecedented (the idea of automatically detecting and reacting to human behaviour has been a popular theme of AI and HCI research for many years). To me the hard part is reliably detecting the patterns of behaviour and defining what those patterns mean in the first place, as well as finding an appropriate balance between user control and system initiative (which will inevitably vary by user, task and context). Rounding off the opening session was Richard Russell from Google, who presented a highly entertaining overview of how Google ads work and the ecosystem that supports them.

Mike Taylor of Microsoft Research

The afternoon session was opened by Greg Grefenstette of Exalead, who presented his thoughts on the intersection of database and search engine technology, and how the two can be combined to create “search based applications”. These provide a unified view of structured and unstructured data, updated in real time and accessible via a familiar search UI (a value proposition not entirely dissimilar to Endeca). One key difference however would appear to be in the emphasis on user experience: the Exalead position seems to be that UX is more of a ‘hygiene factor’, indigenous to the platform itself, and that end users will simply configure the UI however they see fit. But I find it hard to believe that this approach would scale for all but the simplest of applications –I have seen too many examples of poor usability to believe that good design will happen as a fortuitous by-product of end user configuration. Of course, this may all be a matter of interpretation – if you define the scope of UX as starting with query formulation and ending with results presentation, then fair enough; it probably is relatively trivial. But if you define UX that way then you probably have also missed the point.

Vivian Lin Dufour of Yahoo

One of the most memorable talks in the afternoon session was from Richard Boulton, who described his experiences in developing a web service interface to IR code library Xapian. This could have been a relatively dry topic, but was brought it to life by Richard’s personal anecdotes of the eccentric away days and bootcamps he attended in developing the system. The proceedings ended with a panel session on Interactive IR, picking up on the threads that had been generated on the IR mailing list around the subject of the “Google Generation” and the how the information seekers of the future deserve something more than simply a further extension to the best-first ranked results paradigm. The discussion that ensued extended far beyond the original theme, with many delegates joining in to offer their views of how the world’s information might be organised in the future.

The day closed with a drinks reception in the atrium and the IRSG AGM, at which I’m pleased to say many new delegates came forward to serve on the IRSG committee for the forthcoming year. If you’re similarly inspired and interested in joining them, drop me a line. We’re particularly interested in building stronger links with the IR practitioner community. Next year’s Search Solutions will again be held at BCS HQ in late October (date TBC). We’ll be starting the formal planning early in 2010 so if you’d like to be involved just drop me a line either here or at irsg@bcs DOT org. In the meantime, we are hoping to run an Industry Day at ECIR 2010, in Milton Keynes on April 1st. Watch this space for updates!

Read Full Post »

Interactive Information Retrieval at Search Solutions 2009

A while ago I posted out the full line-up for Search Solutions 2009, to be held at BCS London on October 1st. But one thing that we hadn’t finalised then was the panel – that was still very much a TBD, with at least two candidate topics under consideration. So I’m pleased to now confirm that the theme for this year’s panel will be “Interactive Information Retrieval”. It’s a subject close to my heart, and one that is receiving increasing amounts of attention through events such as the HCIR conference series and dedicated columns within UX web magazines such as UXmatters, in addition to growing interest and awareness within the UX and IR communities more generally.

The details for the panel are attached below. I’d like to acknowledge Leonard Will as the (somewhat unwitting) inspiration for this, with his thought provoking observations on the talks given at the BCS event “Search – the Science Making Tomorrow’s World” earlier this year.

Actually, on reflection, the title “Interactive Information Retrieval” isn’t ideal, as it’s a bit generic and doesn’t quite capture the essence of the specific themes alluded to in the abstract. So if you can think of a shorter, better, or more pithy title, please let us know!

It has been widely reported that the “Google Generation” is changing our expectations of online information-seeking behaviour, and that impatience in search and navigation and zero tolerance for delay in satisfying information needs are becoming the norm. But should simple keyword search remain the dominant paradigm for information seeking behaviour in the future? Should we not be encouraging users to employ more sophisticated search techniques than simply entering keywords and relying on best-first ranked retrieval algorithms? What prospects are there for developing a more meaningful interaction with end users, providing a constructive dialogue that supports and facilitates search and information discovery in its widest sense?

Read Full Post »

Card Sorting using the Delphi method

A short while ago I undertook a card sorting exercise for a major eCommerce retailer, involving some 200+ items in their product catalogue. Now, there’s been a lot written about card sorting recently, including some excellent overview articles, so in this post I shall focus solely on the experiences that were unique to our exercise.

In particular, we had to deal with a product taxonomy that:

• contained over 200 product items, arranged across 3-4 levels;
• used 10 top level categories that were relatively ‘fixed’
• was organised mainly by product type, but in some cases by product location (typically the room in which the product would be used);
• was polyhierarchical, in that some content items appeared in multiple places.

Our intention was to run a card sort with ~15 participants (which is considered by some to be the minimum number), with each 90 minute session organised as follows:

1. Perform a brief inverse card sort or ‘findability test’;
2. Perform a closed card sort of the content items using the 10 top-level categories;
3. Elicit feedback from the participants on their ‘Top 10’ most sought items (which could be used as probes for a subsequent findability test)

Card sorting (http://www.usability.gov/design/cardsort.html)

Anyway, here is the twist. A key part of our brief was to review the whole taxonomy, i.e. all 200+ items. For this reason, using traditional card sorting techniques wasn’t going to be practical (at least, not in a regular 60-90 minute session). Instead, we decided to use a relatively new technique known as the Delphi method. In this approach,  the first participant creates an initial sort, and then the remaining participants review and modify that sort. This imposes a much lower mental load on the remaining participants, rendering a sort of 200+ items a much more tractable proposition (in theory). The other part of the brief was to work within the constraints of the existing top-level categorisation, and for this reason we used a closed sort.

The other twist was that we took the opportunity to use both group-based and individual sorting. Now, there are various pros and cons to group vs. individual sorting, such as groups will give you richer qualitative feedback (think-aloud commentary, etc.), whereas individuals will give you more data points (which, depending on your analysis method, can give you greater confidence in the output). But our primary reason for using groups was that the success of the Delphi method is very dependent on the outcome of the initial sort, so running the first few sessions using groups of 3 would reduce the likelihood of an outlier initial sort propagating through the remaining sessions.

So, that was the plan. We ran the card sort itself in a fairly traditional fashion, i.e. content items printed on cards, with participants being allowed to create new cards, set aside existing ones, highlight problematic ones, create new sub-groups etc. What follows below are my reflections on the experience; in particular what we learnt and what we’d change were we to repeat the exercise:

1. The more I think about it, the more I am inclined to agree with Donna Spencer that closed sorts are really only appropriate for narrowly focussed content categorisation tasks. Evidently in our case the client wanted us to analyse all 200+ items including their current top-level categorisation, but my response to this now would be “Is your primary goal to develop an optimal IA, or validate an existing one?” If the former, then an open sort is the way to go, if the latter, then a full-scale findability test seems to be most appropriate. But in neither case does a closed sort seem the right approach.
2. Related to the point above, I think it is so important to begin with the end in mind. I made great use of Donna’s Excel template for card sorting analysis – it not only produces great results but also forces you to think more deeply about your overall methodology. It does have a limit of only 200 cards, which I had to creatively work around in this case, but that limit does reflect a maximum you should attempt to sort in your typical 60-90 minute session (plus some headroom). In hindsight I think a better approach is to find some way of aggregating items so that the overall number becomes more tractable.
3. Did the Delphi method work, in the sense of providing a better IA using fewer resources? I think the jury is out on this. Evidently, I can only generalise so far from just one example of its use. What I would say is that the initial sort is crucial, so it makes sense to use a group for that. But we also witnessed a fair degree of oscillation, whereby one participant would suggest a certain modification, and then the next one would remove it, only for the following participant to put it back. But overall I’m not convinced that by the end we had elicited the same depth of analysis that you’d expect from using a true open sort. It seemed that the participants spent more time reviewing the existing structure than thinking about how they would ideally (re)create it. So my instinct is that an aggregate model generated from a large number of interative open sorts is more likely to produce a more representative IA than the Delphi-inspired incremental approach. That said, Celeste’s Lyn Paul’s paper presents a much more scientific analysis of this issue than my anecdotal assessment, so I think you should read her evaluation before forming any judgement. Bear in mind that we should differentiate also between the approach and the execution – I freely admit that I would do things differently next time around, so we shouldn’t infer from my experience an unduly negative assessment of the technique itself.
4. I mentioned at the start that this taxonomy was polyhierarchical, in the sense that it has the same content item appearing in multiple places. What is the best way of dealing with this? If “product X” appears in four places in the current taxonomy, should I create four cards, thus reflecting the true taxonomy, or just one, and let the statistical analysis show where (and how many times) it should appear? My instinct was to do the former, but I’d like to understand the rationale for choosing one approach over another.
5. Don’t forget to number the cards! It’ll make your subsequent analysis so much easier. Use pencil and write lightly on the back, in such a way as participants won’t notice. And write the label in the middle at the top, so that cards can be nested easily with their labels still visible.
6. Draft appropriate templates in advance for your data collection. With multiple participants you need to automate this as much as possible, so spend some time creating whatever proformas or templates you think are going to be needed. I created templates for recording the outputs of both the findability test and the ‘Top 10’ exercise (in addition to using Donna’s spreadsheet for the card sort).
7. Alternatively, consider using one of a number of online tools for card sorting. Initially I was a little sceptical of these, as most are chargeable services, and with remote testing you’re going to miss out on so much qualitative feedback. But I have been reasonably impressed with Websort – it gets down to business very quickly and greatly simplifies the remote card sorting experience. We’ll be using this tool for our IA work on the Ergonomics Society website.

Read Full Post »

Designing Effective Search and Discovery User Experiences

HCI 2009

On September 4th my Endeca colleague Mark Burrell & I are presenting a tutorial at HCI 2009 called “Designing Effective Search & Discovery User Experiences“. This half-day tutorial provides a  practical introduction to Human Centred Design for information search, access and discovery. We present a concise overview of the “Design for Discovery” framework and show how this can be applied to the design of search user interfaces. A key element of the tutorial is the opportunity to apply these skills  in a practical group exercise.

The cost for this tutorial is just £40, which by most people’s standards is extremely competitive. It certainly compares very favourably with the typical price for commercial user experience training courses, e.g. this one which costs £675 + VAT for 2 days, or this one which costs 425 + VAT for just one day (in fact, I used to teach the first of these two courses). Note that attendees not have to register for the main conference event, i.e. you can attend just the tutorials alone.

See the HCI 2009 conference website for further details. For convenience I’ve included the official blurb below.

Overview

This half-day tutorial provides a practical introduction to Human Centred Design for information search, access and discovery. We present a concise overview of the “Design for Discovery” framework and show how to apply this to the design of search user interfaces. A key element of the tutorial is the opportunity to apply these skills in a practical group exercise.

Benefits

Participants will learn:

• the fundamental concepts and principles of Design for Discovery
• how to differentiate between various types of search behaviour: known-item, exploratory, etc.
• models of human information-seeking behaviour, and how to apply interaction design principles based on those models
• an understanding of the key variables of user type, goal and mode of interaction, and how to apply these variables when designing for varying user contexts
• the role of design patterns, and how to apply Endeca UI design patterns and those of other pattern libraries in designing search user interfaces

Audience

Web designers, information architects, user experience architects, and HCI professionals and researchers interested in the designing more effective user interfaces for search and information access. No specialist skills or knowledge are required.

Instructors

Tony Russell-Rose is User Experience Manager at Endeca Technologies, an enterprise software company specialising in innovative solutions for information search and discovery. Before joining Endeca, Tony was founder and director of UXLabs, a user experience consultancy specialising in technology innovation and applied R&D. Prior to this he was R&D group manager at Canon Research Centre Europe and technical lead at Reuters, specialising in advanced user interfaces for information access and search. He holds a PhD in HCI and a first degree in engineering, majoring in human factors. Tony is also Honorary Visiting Fellow at the Centre for Interactive Systems Research, City University London.

Mark Burrell is Worldwide Lead for User Experience at Endeca Technologies. He has over 25 years of professional experience, including 15 years focused on the evaluation, design, and adoption of interactive technology solutions (with special emphasis on applications that aim to support learning and discovery). Prior to joining Endeca, Mark built and led user experience teams at several leading product and service companies including serving as Sr. UX Manager for Microsoft’s Unified Communications product division and Global UX Lead for Sapient. Mark holds a PhD in Clinical psychology with concentrations in cognitive psychology and epistemology/philosophy of science.

Read Full Post »

New Informer – Summer 2009

We’ve just published the Summer 2009 edition of Informer, which is the quarterly newsletter of the BCS IRSG. This edition features the following articles:

• Editorial by Tony Russell-Rose
• Conference Review: 31st European Conference on Information Retrieval by Deirdre Lungley
• Book Reviews:
  • “Rethinking knowledge management” reviewed by Jennifer Smith and Ben Summers
  • “Safeguards in a world of ambient intelligence” reviewed by Christina Lioma
  • “Search engines, link analysis ans users web behaviour” reviewed by Zheng Zhu
• Events listing edited by Andy MacFarlane
• Industry News, Featured Jobs, Contacts, etc.

We also reveal the lineup for the upcoming Search Solutions event, to be held at BCS HQ on October 1st.

Informer is free to download from the IRSG website. Enjoy!

Read Full Post »

Search Solutions 2009 – lineup announced

It took a while, but I can now finally announce the lineup for Search Solutions 2009, to be held at BCS London HQ on October 1st. For those unfamiliar with Search Solutions, this is an event co-organised by my BCS IRSG colleagues, and we describe it “a special one-day event dedicated to the latest innovations in information search and retrieval”. Which actually doesn’t say that much… instead, this is the key bit:

“In contrast to other major industry events, Search Solutions aims to be highly interactive and collegial (with attendance limited to 60-80 delegates) and features a high quality technical programme. The format of the event will include presentations, panels and case studies in which invited speakers present and share opinions and expertise.”

So while the limit on numbers may have been by necessity when we first held the event, on reflection it is now perhaps more that way by design. Anyway, here is the complete lineup:

• Mike Taylor, Microsoft “Enterprising Search”
  
• Richard Russell, Google  “‘How Google Ads Work‘”
  
• Vivian Lin Dufour, Yahoo TBD
  
• Greg Greffenstette, Exalead  “Search-Based Applications: the maturation of search”
  
• Tamar Sadeh, Ex-Libris  “Librarians, metadata, and search”
  
• Tony Russell-Rose, Endeca  “UI Design patterns for search & information discovery”
  
• Richard Boulton, Lemur Consulting  “Search as a service with Xapian”
  
• Dave Mountain, Placr  “Location-based services: positioning, geocontent and location-aware applications”
  
• Gary Jennings, WebOptimiser  “How and why you need to calculate the true value of page 1 natural search engine positions”
  
• Andrew Maisey, Unified Solutions  “The benefits of taxonomy in content management”
  

We are also honoured to have the company of BCS President Alan Pollard who has kindly agreed to introduce the event for us. And in common with previous years, the main event will be followed in the evening by a drinks reception in the Atrium.

Nick Craswell at SS 2007

One thing yet to be decided though is the topic of the panel. If you attended Search Solutions 2008, you’ll no doubt recall the “Here Comes Everybody” discussion on the wisdom of crowds and how mass participation was the key to scalable information organisation. This was one of the highlights of last year’s event, with enthusiastic participation from all present.

This year, we have a couple of ideas to choose from. One suggestion is to expand on a thread that Leonard Will initiated on the IR mailing list, reflecting on the design of online search and retrieval user experiences and how they still fall so far short of the rich interaction and dialogue we experience when consulting ‘human’ information services. There are many ways in which the latter could be used to inspire the design of the former, and I think this would make a fascinating and topical panel discussion.

Conrad Taylor at SS 2007

On the other hand, we could focus on something that is particular to Search Solutions itself: the fact that we bring together both the web and enterprise search communities under one roof to share innovations and experiences. What best practices and insights can these two communities can learn from each other? I’m sure there are some who’d argue that the answer is less than you would think – but that in itself makes an interesting viewpoint, and the outcome could help steer the direction of the event in years to come.

So, let us know what you think. If you have a preference or an idea of your own you’d like us to consider, then drop us a line either here or via the IRSG inbox at irsg@bcs.org.

Search Solutions is a special one-day event dedicated to the latest innovations in information search and retrieval. In contrast to other major industry events, Search Solutions aims to be highly interactive and collegial (with attendance limited to 60-80 delegates) and features a high quality technical programme.The format of the event will include presentations, panels and case studies in which invited speakers present and share opinions and expertise.

The event should appeal to designers, developers, end users or indeed anyone interested in hearing the latest developments in web and enterprise search.

Read Full Post »

Search UI Design Patterns

Here are the slides for the talk I gave at Create 2009 on UI Design Patterns for Search & Information Discovery. Some of the builds don’t come out quite right on Slideshare, but I can always make the ppt available if people want to see the original. I’m hoping to present an update on this at the forthcoming Search Solutions conference in October.

Read Full Post »