A bug bash is a session with a fixed time limit, usually one to two hours, where people from across a team use a near-final build at the same time to find problems before release. It differs from normal QA testing because the people in the room include more than testers, and the goal is to cover the ways real people use the product rather than to follow a written test script.

Who should join a pre-launch testing session besides engineers?

Include the people who will support the product after launch and the people who never worked on the task, such as sales, customer support, someone from finance or operations, and anyone on the team who fits the target user profile but was not involved in building the feature. Engineers who wrote the code already know the intended steps, so they are the least likely people in the room to get confused the way a new user will.

Why do group testing sessions usually find typos instead of real usability problems?

Because an open-ended "click around and find bugs" instruction lets people report the easiest thing to spot, which is a misspelled label or a misaligned button. Giving each participant a specific task written as a goal, not a set of steps, forces them to find their own way through the product, which is where confusing steps and missing screens actually show up.

How long should a group testing session run?

Most effective sessions run one to two hours. Long enough for a participant to complete two or three realistic tasks without knowing the product, short enough that attention does not drop and people do not start working around problems instead of reporting them.

How many participants does a group testing session actually need?

Research from the Nielsen Norman Group found that testing with five users typically finds about 85 percent of a product's usability problems, because most problems repeat across users while only a few are unique to one person. A pre-launch session with six to ten people, each given a different task, covers more of the product than a larger group with no plan, all clicking through the same normal use.

What is the difference between a group testing session and QA testing?

QA testing follows a written test plan against expected behavior, usually run by the same people every release. A group testing session removes the script and adds people who were not involved in building the feature, so it catches the gap between what the team assumes a user will do and what an unfamiliar person actually does when given a goal and no instructions.

Should the same person sort every finding from the session?

No single person should have sole authority to downgrade a finding, because the person closest to the code is also the person most likely to see a real gap as a small detail. The sorting should include at least one person who was not on the build team, and every downgrade decision should be written down with a reason next to the status change.

What should happen to a finding from the session that gets deferred?

It needs an owner and a date as well as a lower priority label. A finding with no owner and no date functions the same as a finding nobody wrote down, because nothing forces anyone to return to it once launch week ends and attention moves elsewhere.

Can a group testing session replace automated testing?

No. A session like this finds problems in how a real person understands and moves through the product, which is not something an automated check can evaluate. Automated checks are better at catching a regression the moment it is introduced and at re-verifying the same rule on every future change, which a one-time session cannot do. The two catch different types of problem and neither substitutes for the other.

What is the most common mistake teams make running a group testing session?

Treating it as an informal event with no plan instead of a structured session. Without assigned tasks, a time limit, and a person responsible for sorting findings, the session becomes an unstructured hour where people report whatever they notice first, which is almost always spelling and spacing, not the flow that will actually confuse a paying customer in week one.

How to run a bug bash before launch: Reveneau

Q: How do you decide what gets fixed before launch versus after?

Sort every finding into one of three groups: it stops or damages the main task, it is confusing but has a workaround, or it is cosmetic. Only the first group must be fixed before launch. The second group goes on a dated list for the first post-launch update, and naming that date in the meeting is what stops an "important" finding from being argued back into the first group anyway.

We have sat in bug bashes (sessions where many people test a near-final product together before launch) that produced forty tickets and not one of them mattered. Every ticket was a typo, a color that did not exactly match, a button that could be two pixels to the left. The room had spent an hour together and found nothing that would have stopped a real customer from getting stuck. The product had real problems. The session was set up to find small things instead of real ones, and that is a structural problem with three fixes: who is in the room, what you ask them to do, and how you decide what gets fixed before launch versus after.

Invite the people who did not build it

The person who wrote the code is the worst person to test whether a stranger can use it, because they already know where everything is.

A session like this exists to have new people look at a near-final build before it reaches a real customer. The practice traces back to Ron Patton's book Software Testing, which described gathering developers, testers, and other staff to test a build together outside their normal roles, and it became a standard pre-release step at companies including Microsoft and Google, where cross-team sessions are still run before major releases.

For that to work, the room needs people who were not on the build team. Invite customer support, who will answer the calls when something breaks. Invite sales, who know what the customer was promised. Invite someone from finance or operations who fits the target user profile but has never used this feature. None of these people know the intended steps, so they cannot follow them by accident. That is the whole value. An engineer who built the checkout flow will type the right values into the right fields in the right order without thinking about it. Someone from support, given the same screen and a goal instead of a script, will run into the same problem a real customer will.

Keep engineers in the room too. Somebody needs to be able to say whether a report is a real defect or expected behavior, and to fix the worst ones immediately. Just do not let them be the majority.

Give people a goal instead of a checklist

Most of these sessions fail because of the instructions people are given.

"Click around and try to break it" produces typo reports, because a typo is the easiest thing to notice and report, and an open-ended instruction rewards whatever is easiest. A specific task rewards something different. Write each task as a goal the way a customer would describe it, not as a list of steps: "You just signed up. Get your team invited and your first project set up." "You are trying to cancel your subscription before the trial ends. Do it." Do not tell people which screen to start on or which button to press. The difference between the goal and the steps they actually take is where the real findings are: a confusing label, a missing confirmation, a state the interface never explains, a screen with no way forward and no way back.

Split the group so different people run different tasks, instead of everyone repeating the same normal flow at the same time. Six people repeating the same signup flow will find the same three problems six times. Six people running six different realistic tasks, chosen to cover the flows that matter most, cover six times as much of the product.

Run the session for one to two hours. Long enough for someone unfamiliar with the product to attempt two or three tasks without rushing. Short enough that people stay focused and report the problem instead of quietly working around it, which is what tends to happen once people get tired.

On sample size: the Nielsen Norman Group's research found that testing with five users typically finds about 85 percent of a product's usability problems, because most usability problems are common enough that several users run into the same one, while only a few are rare enough to need a much larger group to catch. A session with six to ten participants, each assigned a different realistic task, is closer to that finding than a bigger group clicking through the same one or two flows. More people help less than different people.

Sort the findings in the meeting, immediately, with a rule

A session like this produces many findings fast, and the list itself is not the result. What matters is what gets fixed before the product is released and what gets scheduled for after.

Sort every finding into one of three groups, and do it in the meeting while everyone still remembers the details, instead of in a ticket queue two days later:

Stops the main task. The user cannot finish what they came to do, data is lost or corrupted, or the product does something it should never do (charges twice, deletes without confirming, exposes another account's data). This must be fixed before launch. Nothing else needs a debate.
Confusing but has a workaround. The user can finish, but it took longer than it should have, or they had to guess. This is real, and launch can go ahead with it. It goes on a list for the first post-launch update, with a name and a date added in the meeting, instead of "soon."
Cosmetic. Spacing, color, text that could be shorter. Fix it if it is fast. Otherwise it waits.

The naming and the date matter more than they sound like they should. A finding that gets marked "important" and then added to a list of later work with no owner will not get fixed, and everyone in the room knows that, which is exactly why people keep arguing that their finding belongs in the first group even when it does not. Give the second group a real date for a real update, said out loud, and the argument mostly goes away, because deferring a finding no longer feels like losing it.

Do not let one person, especially the engineer who built the feature, be the only judge of what gets moved from the must-fix group to later work. The person closest to the code is also the person most likely to see a real gap as a small detail, because they already understand why the interface behaves the way it does. Somebody in the room who does not have that context should have a real vote on the sorting, and every downgrade should get a one-line reason written down next to it. That line is what keeps the second group from becoming a list where real problems are forgotten.

What a group testing session cannot do

A session like this finds problems in how a real person understands and moves through the product on one specific day, with one specific build. It will not catch a regression introduced next month, and it will not re-check the same rule on every future change the way an automated check does. In the work we do, a testing session and an eval suite answer different questions: the session tells you whether a stranger can complete the task, the automated checks tell you whether the thing that worked yesterday still works today. Running only one of them leaves a real gap.

That is also why a testing session belongs right before launch, and why it cannot replace testing earlier. It is expensive to run, it needs real people's real time, and it works best on a build that is close to final. Automated checks run on every change, all the way through development, because a machine's time is nearly free and a person's is not.

Thanks to the customer support and sales teams who have shown up to these sessions with no reason to attend, and found the exact problems that the build team knew the product too well to notice. The fastest way to learn what confuses your customer is to hand your product to someone who has never seen it and watch, quietly, where they get stuck.

Sources

Nielsen Norman Group, Why You Only Need to Test with 5 Users: testing with five users typically finds about 85 percent of a product's usability problems, because most problems are common enough to repeat across users.

How to run a bug bash before launch