How to Beta Test

One of the trickiest things to do at a law firm is to test a new technology product. When done right, the result is a new piece of hardware or software that makes life easier for attorneys. When done wrong, it can result in muffled curses emanating from behind closed office doors, and money wasted on a new gadget that no one ever uses.

That's why smart firms tend to approach beta testing like a military operation, meticulously planning the test in successive waves that build on small victories before launching a firmwide attack. At Gibbons PC, a firm with 230 attorneys in five East Coast offices, the technical team makes a few fundamental tactical assumptions before the formal beta test ever begins.

"A product rarely comes out of the box ready for your users, or the way the vendor says it will work," says Michael Aginsky, chief technology officer at Gibbons. "Everyone oversells and promises you everything."

When the Gibbons team is presented with a new product or service its attorneys have approved, it first analyzes the expectations that accompany the new product. What were the attorneys sold on? What do they plan to do with the new product? And how shall the technical team configure it to look and perform the way attorneys expect it to?

Gibbons is in the early stages of beta testing Microsoft SharePoint, a popular collaboration tool that lets businesses manage, search, and share company information and documents securely via a Web portal or intranet. Gibbons is particularly keen to migrate the firm's present Web worksite over to a SharePoint portal, but at least three months of beta testing are needed before implementation. SharePoint, while highly regarded as a powerful collaboration portal, is notorious in technical circles for needing a lot of configuration and customization before it conforms to a firm's natural work flow.

"You have to figure out who should suffer the most in the beta test," says Patrick DiDomenico, Gibbons's chief knowledge officer. "And from our perspective, [CTO] Michael Aginsky and I are going to do most of the initial suffering. Even before you get to a small beta group of attorneys, it's important to take the bruises out of the product. So we'll do some pre-beta testing?just me and Mike?and get it to a point where it's work-able. Then we'll look for the pain points?see what works, what doesn't."

Gibbons has no formal technology committee, but it often pulls together a user group of lawyers, secretaries, paralegals, and case managers to give further suggestions about how to roll out the product to attorneys.

A group of about 30 users, including attorneys and administrative groups, will eventually conduct the formal beta test. But the tests start with just a handful of users, and then gradually more people are invited into the testing group. This staged rollout of the beta test lets the technical team concentrate on individual problems and user concerns rather than fielding a blizzard of questions from dozens of baffled beta testers at once. SharePoint is then tweaked and customized along the way, adding features when necessary, but always according to the Golden Rule of beta testing: Keep it simple.

"Whatever product we're rolling out, I always want it to work like the iPod," says DiDomenico. "It should be elegant, intuitive, and just work."

---