Open Card Sort Analysis

Open card sorting analysis has two main goals:

1. Determining the best group names
2. Determining the best card placement

Depending on the diversity of group names, it can be beneficial to follow up an open card sort with either a closed card sort or a Tree Test to validate that the chosen names work for your information architecture.

Getting your bearings with the Participant View

To begin, it's often a good idea to review the Participants view. Using this view there are a few things to keep an eye out for.

1. By sorting on the Time column you'll see the participants in the order of task duration. Participants who spent a far below average time completing your task could indicate there wasn't much thought put into their groupings. You can use the Results link at the far right of the table to review their individual results and decide if you'd like to exclude them from your results.
2. If you enabled Allow Unsorted Cards in your card sort settings, you'll see a Cards column. Here again is a good checkpoint to possibly exclude users who didn't put the appropriate amount of effort in and have a low number of cards sorted.
3. Sort on the # Groups column. If you see a participant who sorted an unusually large or small number of groups this could indicate they are an outlier of your results. Keep this in mind as you work through group merging.

If you decide to exclude a participant, make note of their identifier and exclude them in the Projects Participants view.

First Stop : Cards View

The first step of an open card sort is to digest the group names and begin to do group merging. Group merging is done on the groups view, but it is often helpful to begin on the cards view to get sense of the various group names participants used.

Let's use an example open card sort where users were asked to sort movies into categories. You can see below that there are several group names for Pulp Fiction that revolve around violent (go figure). There is also multiple uses of many other words such as classic (cult classic,classics to watch). Making note of these types of similarities will help in the next step of group merging.

Note: we do not automatically merge groups names with differences in capitalization. This is done to maintain group names with acronyms.

Merging groups

After an initial look at the cards table you should have some ideas of what to merge. You can begin merging the easy ones. In our case this is things such as Violent, Violent Action and violent. For details on how to merge see our section on merging groups.

Then transition into more difficult decisions about group names that may be synonyms you want to merge. For example. One person put Pulp Fiction in Thrillers, while 6 people placed it in Drama. Looking at the groups table you'll be able to see other cards that were placed in each of these groups to help decide if your audience viewed them in the same way. Through out this process, each decision to merge is a judgement call on your part.

Back on the cards view after a first round of merging, the number of unique group names has gone from 19 to 12. In the Group Name column you'll see underlines to represent groups that are merged.

At this point you'll continue to merge groups. As you refine, you may begin to see outliers that raise concern for the validity of an individuals results. As an example in this card sort: Children was given as a group name for Pulp Fiction. In this situation, I would find the user who provided that group name in the Raw view, review their other choices and make a decision about including them or not.

Strong pairs & weak pairs

At this point it's sometimes helpful to consider the relationship that each card has with other cards. The similarity matrix shows the frequency that two cards were grouped together. Identifying strong clusters can help further your view of a proposed final architecture. Use the merge group toggle to see the impact your merging has on the strength between cards.

In our example open card sort you can see that Toy Story & The Incredibles were grouped together 85% of the time.

Considering the number of final groups.

In an open card sort the tree view can be used as a guide to how many groups your final architecture should have. By visualizing vertical lines going through the tree at various points and counting the number of intersections you can determine group counts. In this example below, the red line drawn intersects 5 times which could be used to guide the number of final groups.

Iterative refinement & other considerations.

This process of understanding the card placement, merging groups and checking the results continues until you've reached a satisfactory level of confidence to declare a final information architecture.

If you've included a survey with demographic information consider that you may have multiple user groups who think about your content differently. Use survey filters to evaluate your card sort results for each group.

Next Steps

After an open card sort it can frequently be good to follow up with a closed card sort, or a Tree Test to validate your chosen group names and card assignment match the users mental model and will be able to find the content on your site.