Mining search logs for usage patterns (part 2)

In a previous post I discussed some initial investigations into the use of unsupervised learning techniques (i.e. clustering) to identify usage patterns in web search logs. As you may recall, we had some initial success in finding interesting patterns of user behaviour in the AOL log, but when we tried to extend this and replicate a previous study of the Excite log, things started to go somewhat awry. In this post, we investigate these issues, present the results of a revised procedure, and reflect on what they tell us about searcher behaviour.

Read Full Post »

Mining search logs for usage patterns (part 1)

As I mentioned in a previous post I’ve recently been looking into the challenges of search log analysis and in particular the prospects for deriving a ‘taxonomy of search sessions’. The idea is that if we can find distinct, repeatable patterns of behaviour in search logs then we can use these to better understand user needs and therefore deliver a more effective user experience.

We’re not the first to attempt this of course – in fact the whole area of search log analysis has an academic literature which extends back at least a couple of decades. And it is quite topical right now, with both ElasticSearch and LucidWorks releasing their own logfile analysis tools (ELK and SiLK respectively). So in this post I’ll be discussing some of the challenges in our own work and sharing some of the initial findings.

Read Full Post »

A taxonomy of search sessions

Over the last few months I have been working with Paul Clough and Elaine Toms of Sheffield University on a Google-funded project called ‘A Taxonomy of Search Sessions’. A session, in case you’re wondering, is defined as a period of continued usage between a user and a search application. So if you spend a while Googling for holiday destinations, that’s a session. Sessions are interesting because they form a convenient unit of interaction with which to study usage patterns, and these can provide insights that drive improved design and functionality.

Read Full Post »

Designing Search (part 1): Entering the query

In an earlier post we reviewed models of information seeking, from an early focus on documents and queries through to a more nuanced understanding of search as an information journey driven by dynamic information needs. While each model emphasizes different aspects of the search process, what they share is the principle that search begins with an information need which is articulated in some form of query. What follows below is the first in a mini-series of articles exploring the process of query formulation, starting with the most ubiquitous of design elements: the search box.

1. The search box

One of the fundamental concepts in HCI is notion of affordance: the idea that objects should behave in the manner that their appearance suggests. A push plate on a door affords pushing; a handle afford pulling. How many times have you walked up to a door and found it behaved contrary to your expectations? Invariably this is caused by a mismatch between form and function.

Likewise, the design of the search box should follow its function. If its purpose is to allow the user to enter queries in the form of keywords, then it should look like it will accept textual input, and have an associated button which clearly indicates its function. It should also be wide enough to comfortably accommodate the majority of queries:

A match between form and function at eBags.com

The examples below, by contrast, are perhaps somewhat less effective:

Read Full Post »

User research plans for ES website

The Ergonomics Society is about to embark on a redesign of its website, and ealier this month I posted out the initial user segmentation model, along with the draft user profiles and the prioritised scenarios. Now, following  conversations with various folks including Tina Worthy and Richard Bye, we have an updated plan for user research.

In summary, what we plan to do is:

1. Establish some baseline data for the existing site experience (so that we have something to compare with after the redesign). Richard Bye has kindly offered the use of his analytic tools in assessing this.
2. Perform depth interviews with participants from the 1st four priority segments, as follows:
   • Information Consumers (times 3)
   • Society Members (times 3)
   • Society Customers (times 2)
   • 3rd Party Service Consumers (times 2)
   1. Note that the breakdown here is designed to reflect both the relative priorities of the segments and what we feel is realistic given the resources available.
3. Hold a focus group for the Staff Information Consumers.
4. Run a formative IA exercise (such as an open card sort) to establish the key organisational principles for the site content. Participants to be segmented as in (2).

Evidently, there will be a fair amount of prep involved in all of this, notably the preparation of recruitment screeners, interview protocols, scripts, etc. Note also that the analytic tools that Richard has offered will also need configuring; no doubt a key part of this will be determining precisely what metrics to measure as a baseline. I suspect we’ll need to adopt a pretty lightweight / agile approach, especially considering that most if not all of this will need to fit around existing work commitments. And we shouldn’t underestimate timelines either – it is one thing to manage delivery of a web project when everyone is directly accountable to you; quite another when everyone is lending their time on a voluntary basis.

Looking further ahead, we will also need to consider the choice of development platform.  At the moment we are using phpMyAdmin, but it is likely that we will want to migrate to something more scalable and usable by a wider cross section of people (i.e. nominated content editors) in future. Lauren Morgan is currently evaluating alternatives such as Joomla and Drupal, and should be in a position to report back soon.

So, as a rough estimate, I’d say the timeline will pan out something like this:

• August: user research
• September: user research + data analysis. Output = refined segmentation model + profiles + scenarios
• October: Interaction design + visual design (proceeding in parallel in so far as that’s practicable). Output = wireframes (which could be fairly simplistic, depending on the build approach) + visual design spec. (NB we should also consider producing a style guide for the site, but I am not sure we can deliver that as well within the scope of the exisiting project)
• Nov + Dec: build. Output = CMS templates + associated tools & resources, etc.
• Jan: UAT + soft launch
• Feb: full launch

Note that I’m assuming we will interveave user feedback at suitable iteration points throughout the above timelime – as UCD specialists we should know this better than any 🙂

Read Full Post »

Audience segmentation model for Ergonomics Society website

The Ergonomics Society is about to embark on a redesign of its website, and on Thursday last week we held the stakeholder kickoff meeting. The purpose of this meeting was threefold:

1. To bring together the key stakeholders to establish a common vision for the project
2. To develop a basic UX framework upon which subsequent design work would be based
3. To establish a baseline for subsequent project planning and resourcing (including any user research requirements).

We had only three hours to achieve all this, so there wasn’t much time for prolonged discussion. Present at the meeting with me were Dave O’Neill (CEO), Sue Hull (Conference Manager), John Winter (Membership Development Manager), Kia Horrocks (Conference & Membership Administrator), Tina Worthy (Web Editor), Lauren Morgan and Lauren’s colleague Maya (student rep).

First, we took a minute to write down what we thought should be the vision for the Society website, before sharing these as a group. It turns out that our individual visions were remarkably consistent, with most including some notion that the site should be both a comprehensive information resource for the profession as well as an advert or showcase for UCD. For example: “to be the information resource for ergonomics and human factors profesionals and to be a showcase for great UX design”.

The next activity was a group session to establish the key audience segments for the wesbite. It transpired that there is little in the way of previous user/market research available, so we had a fairly blank state to begin with. After  considerable whiteboarding and iteration, we finally identified the following key segments with their respective priorities (where P1 is highest, P3 lowest):

• P1. “Information Consumers”, e.g. researchers (both individuals and corporate), academics/teachers, press, advocates of ergonomics / human factors, etc.
• P1a. Society Members, i.e. anyone who is currently classed as a member (on any of the grades)
• P2. Society Customers, i.e. anyone paying for Society products & services, such as conference delegates and advertisers
• P2. “3rd Party Service Consumers”, i.e. agency clients, short course attendees, and prospective ergonomcs students (i.e. anyone engaging services via the paid listings)
• P2 “Staff Information Consumers”, i.e. same as the first group but employees of the Society
• P3 Website Editors, i.e. Society staff responsible for web content, such as PR / Comms, membership etc.
• P3 Website Developers. i.e. Society staff responsible for web development

You might be wondering why “Information Consumers” was given a higher priority than “Society Members”. The answer lies in the Vision statement, which suggests that the primary purpose is to serve the profession more generally, rather than those individuals who happen to be members at any given time. (On reflection, this decision – or my interpretation thereof – may benefit from a little further examination.)

BTW, during my time as a UX PM at Microsoft we used the expression “eating your own dogfood” to describe the practice of adopting the products or techniques that you promote. Likewise, I am keen that we “eat our own dogfood” in this exercise, and are seen to be applying UCD best practice within our own web design & development project work. Comments and feedback on both the process and the deliverables are therefore more than welcome.

In the meantime, I’ll be working on transcribing the draft profiles we developed for the priority segments above. More soon – watch this space.

Read Full Post »