Closed Card Sort Analysis

With closed card sorting, the goal of your analysis is to determine the final placement of the cards within the pre-defined categories. The analysis of closed card sorts is typically easier than open or hybrid because there isn't any reconciliation of the group names.

Getting your bearings with the Participant View

To begin, it's often a good idea to review the Participants view. Using this view there are a few things to look out for.

1. By sorting on the Time column you'll see the participants in the order of task duration. Participants who spent a far below average time completing your task could indicate there wasn't much thought put into their groupings. You can use the Results link at the far right of the table to review their individual results and decide if you'd like to exclude them from your results.
2. If you enabled Allow Unsorted Cards in your card sort settings, you'll see a Cards column. Here again is a good checkpoint to possibly exclude users who didn't put the appropriate amount of effort in.

If you decide to exclude a participant, make note of their identifier and exclude them in the Projects Participants view.

Beginning with the Maximum Agreement View

The first view we recommend looking at is the Maximum Agreement view view. While some may view it as the solution, it vary rarely tells the whole story. However, it is a perfect first step to understanding your card sort data.

The Maximum Agreement view displays a compiled results set where each card is displayed in the group where the most participants placed it. The percentage of time the card was placed in the group is displayed to the right of the card name.

This image shows that the card 'Potato' was placed in 'Vegetables' by 84% of participants.

When a card is placed into different groups at equal percentages the card is displayed in ALL of those groups and the percentage is highlighted. Pay special attention to these cards because you will need to work through breaking the ties later in your analysis.

While digesting the data from this table, the agreement value should be carefully considered. The lower the agreement, the less confidence you should have in the cards placement. An agreement of 60% means means that 40% of the participants didn’t put the card in that group. In instances like this it is necessary to understand the other places the user has placed the card.

Diving deeper with the Groups x Cards View

The Groups x Cards view tends to be one of the most powerful views for closed card sorts. The table displays the percentage of time a card is placed in a group. The first column shows the list of cards, and the remaining columns show the group name in the header. A value of 100 means that all participants placed the card in the group. This view is especially useful when understanding all the places that a card was placed when the maximum agreement is low.

Click the column headers to sort the cards in a specific group.

Using this example you can see that the first five cards have greater than 75% agreement in being a vegetable, and after that things decline rapidly. One of the worst cards is 'Olives'. You can see that 'Olives' was placed 53% of time in 1 groups, followed by 37% and 10% in the remaining.

Resolving more difficult card placement

Our example shown above is quite simplistic. But what if you have 10 groups, and the distribution is almost equal? In some situations people will naturally think about things differently, but there are a few things you can do to resolve potential conflicts.

1. Dig into survey data - If you included a survey with your card sort you may have the answers you need in your questions. Frequently different user roles, jobs, or other demographics will shape the results. Experiment with adding filters to your survey answers to help zero in on your most important demographic.
2. Re-think your group name - Depending on how you arrived at your group names it may be necessary to investigate if there is a better name. If this is an option, consider running a smaller set open card sort to work through the naming again.
3. Consider user roles - Depending on your content structure and website, there may be user roles who do not have a need for specific cards. Understanding those roles may guide you to resolving conflicts.
4. Card Names - It may be that the card name itself was ambiguous or not understood.
5. Reach out to your users - In some situations nothing is going to replace a conversation with your users to understand their mindset and gain more understanding.

Other views that may add insight

While we've focused on the most beneficial views of your data for a closed card sort, there are additional views that may help round out your understanding of your results.

Cards View

The cards view is perfect for understanding which cards had the most ‘unique groups’. The first column of this view will tell you how many unique groups the card was placed in. Sorting the column you can quickly find your problem cards - those cards that your participants placed in different places.

Groups View

The groups view complements the Cards view by showing you the same information from the groups perspective. Looking at the agreement column you’ll see which groups your participants had good agreement about the cards it should contain. The higher the number, the better agreement.If you enabled ‘Group Commenting’ in your sort, this is also where you will see those comments.