Card sorting and tree testing: building the menu around users

What is card sorting

In card sorting, each card carries the name of an item of content on your website. Participants sort the cards into groups that make sense to them and name the groups. From these groups you work out what your menu could look like.

The aim is to see users’ mental model. Teams often organise content by the org chart; users search by the question in their head. Card sorting makes the difference between these two orders visible.

The 2024 NN/g article by Samhita Tankala and Katie Sherwin splits the work into four steps. First you prepare the cards, then you ask participants to group the cards and name the groups. In the last step you analyse the data.

Participants form the groups first and only then give them names. This order doesn’t push them into ready-made categories.

In-person sessions use paper cards; remote studies use online tools. According to NN/g, remote studies are more common because the tool summarises the groups itself.

Which content becomes cards? Start with the content users look for most often. Site search logs and questions to customer service give a good first list.

Each card describes a single item of content and is short. Write the card name so that it describes the content itself, not like a menu label.

Card sorting is the research part of work on information architecture. It comes before the menu and category names are settled.

Open, closed and hybrid card sorting

There are three types. The difference lies in who forms the groups.

Type	Groups	When
Open	Participants create and name them themselves.	When you want to know how users think about the content.
Closed	You set them; participants sort the cards into them.	When the group names are fixed and you want to see where the content lands.
Hybrid	You set some; participants can add new ones.	When you are really sure about one or two groups.

In open card sorting, participants see no ready-made categories. They form the groups themselves and name them in their own words. These names are good raw material for menu labels.

In closed card sorting, you set the categories and participants only sort the cards. Some teams use this to confirm an existing menu. For evaluating menu categories, though, NN/g recommends the tree test.

Hybrid card sorting mixes the two, and NN/g usually advises against it. The reason: preset categories push participants to form groups with a similar structure. It is only worth considering when you are really sure about one or two categories.

Then there is the session format. In a moderated session, a researcher sits in and can ask: “Why did you put that there?” In an unmoderated study, participants use the tool alone.

If you want qualitative insights, choose moderated sessions. For numbers, an unmoderated study works better.

Try it: a mini card sort

Below are eight items from a made-up city website. I set the groups, so it is a closed card sort. Choose a group for each card; when all eight are sorted, a summary compares your grouping with mine.

Mini card sort

Groups: Pay, Apply, City life.

1. Property tax

2. After-hours pharmacy

3. Building permit

4. Resident parking sticker

5. Events at the community center

6. Sewer bill

7. Marriage license

8. Bus schedule

My grouping: Pay: Property tax, Sewer bill. Apply: Building permit, Resident parking sticker, Marriage license. City life: After-hours pharmacy, Events at the community center, Bus schedule.

8 items from a made-up city website as a closed card sort. I set the groups; you sort the cards. When all eight cards are sorted, a summary shows how many cards are in the same group as mine. My grouping isn’t a right answer but a point of comparison. The card names share no word: NN/g writes that participants group cards with the same word by that word.

Notice the “Resident parking sticker” card. You pay for the sticker, but first you apply for it, so both groups can be argued for.

In a real study, cards like this are the most valuable finding. An item on which participants split may deserve links from two places in the menu.

What is a tree test

A tree test checks your menu tree without visual design and without content. Participants see only the category names. You give a task such as “You want to pay the property tax on your house”, and the person clicks through the tree to find the right place.

According to Page Laubheimer of NN/g, the tree test measures whether important content can be found in the category structure. It also shows whether the labels are understood.

Cues such as colour and page layout don’t come into it. So structure and words alone decide the result.

Preparing the tree. Don’t write down only the top menu. Add every branch down to the lowest level where the content being looked for sits. Laubheimer recommends a spreadsheet for this: the home page in the first cell of column A, the sublevels in the columns to the right, one category per row.

Writing the tasks. Don’t use the word from the menu label in the task, or you give the answer away. If the menu says “Sewer bill”, the task could be: “You received a notice from the city about wastewater and want to pay the amount.”

Keep the scenario short, because participants miss details in a long story. Put an easy warm-up task first and settle the correct answer for each task in advance.

What you measure. The success rate is the share of participants who find the right place. Directness is the share who get there without backtracking. Then there are task time and first click.

Which method when

In short: card sorting generates structure, and the tree test checks it. Laubheimer describes card sorting as a generative method and doesn’t consider it particularly suited to evaluating a menu hierarchy. He recommends always testing a hierarchy that came out of card sorting.

Your question	Method
How do users group this content in their heads?	Open card sorting
Are the menu labels understood, and is what people look for found?	Tree test
Is the new menu better than the old one?	Tree test with two trees, separate participants per tree
Why did users form these groups?	A few moderated card sorting sessions
Do the layout and visual cues work?	Usability test

The order is usually this. First you collect users’ groups and words with open card sorting. From them you build one or two drafts of the tree.

Then you check with a tree test in which draft content is found more easily. Only then do you move on to wireframes.

At the start of a redesign, you can also check the existing menu with a tree test. Laubheimer recommends this as a baseline. It shows which tasks are struggling and later lets you prove how much the new structure improves things.

The tree test also helps you not to fixate on the number of clicks. The three-click rule sees depth as the problem. In a tree test, the real question is whether the person chooses the right branch at each step, and directness turns that into a number.

How many participants you need

Card sorting. The 2024 NN/g article recommends at least 15 participants for a qualitative study and at least 30–50 for a quantitative one. For the number of cards it also recommends 30–50.

With more than 50 cards, participants may form one big “Other” group. Then you can’t tell whether the person was confused or tired.

There is data behind the number 15. In 2004 Jakob Nielsen reported on a study by Tom Tullis and Larry Wood. The two drew smaller random samples from the results of 168 users.

With 5 users, the results correlated only 0.75 with the full data. With 15 users the value rose to 0.90, with 30 to 0.95. Nielsen therefore recommends 15 users and considers 30 reasonable for large projects with a budget.

This recommendation differs from the rule of testing with 5 users. A usability test evaluates, and big problems often show with just a few people.

Card sorting, on the other hand, is generative. How people order terms and which words they use varies a lot, so it takes more people.

Tree test. For a qualitative tree test, a few participants are enough according to Laubheimer; a quantitative one needs a larger sample. If you compare two or more trees quantitatively, you typically need 50 or more participants per tree.

When comparing two trees, assign each tree its own group and use the same tasks in both. Laubheimer recommends this set-up, in which each person sees only one version of the tree.

How to read the results

In card sorting, online tools summarise the results for you. One of the important views is the similarity matrix, which shows how often two cards are grouped together. Cards that are often together are candidates for sitting next to each other in the menu too.

Numbers say what belongs together but not why. NN/g recommends moderated sessions for qualitative insights, because follow-up questions are possible there. The reasoning in participants’ own words helps you choose group names.

In a tree test, don’t read the success rate alone. Laubheimer cites data from Albert and Tullis from 98 studies, where the median task success is 62%.

By that grading, values under 40% count as poor, 61–80% as good and over 90% as excellent. For critical tasks, the goal is ideally over 90%.

If success is high but directness low, participants find the place but by detours. If first clicks are right but success is low, the subcategories resemble each other and you need to separate the labels more clearly.

If first clicks spread over several areas, the content may belong in more than one group. If that repeats over several tasks, you should rethink the main structure.

If an item was looked for in two categories, it is tempting to place it in both. Laubheimer is cautious about this and recommends it only when a sizeable share of participants looks there from both parent categories.

Decide based on the data on destinations. If one of the correct places is rarely chosen, remove that second link.

Tree testing tools usually don’t calculate whether the difference between two trees is significant. Laubheimer writes that you have to do that calculation yourself. Run the significance test before presenting a small difference as “The new menu won”.

Common mistakes

Shared words in card names. Participants may group cards such as “Pay property tax” and “Pay sewer bill” by the word “pay” instead of by content. That is why NN/g recommends avoiding identical words on different cards.

Turning the card sort result straight into the menu. Card sorting only gives groups on one level, not subcategories. Check the structure with a tree test before it goes live.

Too many cards. NN/g recommends 30–50 cards to avoid fatigue. Pilot the cards with a small group first and make sure the number fits and the labels are understood.

Tasks in the team’s words. If the menu label itself is in the task, participants only match words. Then you aren’t measuring findability.

Working only unmoderated. Laubheimer recommends at least a few moderated pilot sessions. Unclear task wording shows up there before you collect the real data.

My note

The most common information architecture problem I see in audits: the menu mirrors the organisation’s internal structure. On a city website, department names sit in the menu; at a bank, the names of product teams. Users, though, ask themselves: “Where do I pay my outstanding bill?”

For small teams, a full card sorting study doesn’t always fit. Then I recommend a short tree test of the existing menu first. Five or six tasks and a few moderated sessions usually show where the menu breaks.

I count cards like “Resident parking sticker” in the demo as findings. An item on which participants split will very likely cause discussion in the team too. Settling that discussion with data is faster than in a meeting.

These methods don’t test the screen itself. A menu can work well in the tree and look bad on a phone, so after the structure I test the design again with real users.