City Paper is not for tourists
The Zagat guide turned 30 years old this year, and in honor of the occasion, I’d like to give founders Tim and Nina Zagat a hearty thanks for all their years of service to the restaurant industry. And, if I may, I’d like to offer some friendly advice, too: You can go away now.
Ten years ago, back before everyone had access to a world of opinion via some device tucked into purse or pocket, diners relied on your slim red eponymous guide, which compiled restaurant ratings based on public opinion. It was your slap against the imperial voice of the critic. The Zagat book would be the voice of the people, guiding diners by the collective wisdom of the people, even if you would never tell us how many people actually voted for any one restaurant.
But now, nearly a decade into the 21st century, the people no longer need Zagat to compile data, crunch it, and cough it up in cute little numbers of dubious quality. They can compile their own dining information of dubious quality without all the opaque, Wizard of Oz, man-behind-the-curtain nonsense that you have insisted on for three decades.
The truth is, the Zagat guide belongs to a time when tourists and urban newcomers, looking for some guidance on restaurants, had nowhere else to turn. For these folks, the Zagat guide was like asking 1,000 random strangers, “What’s your favorite restaurant?” and then tabulating the results.
Except it wasn’t. The Zagat survey has never been random. Its respondents are, in the language of pollsters, self-selected. Or perhaps mostly self-selected. That’s the problem: Few people outside of Zagat’s New York offices know who actually votes and how those voters are selected.
Nina and Tim Zagat, a pair of former lawyers, have indicated in the past that they’d send out hundreds of thousands of surveys to law firms, medical offices, and other white-collar institutions where people, presumably, have the disposable income to eat out often enough to provide the Zagats with the free data they need to sustain their empire. But Zagat also solicits online reviews. Are those included in the final ratings? That would seem to be the case, but one restaurateur complained to me that some never appear on the site.
The Zagats are like a couple living in a walled compound, sealed off from some of the major developments of the past decade. The generation that willingly accepted unverified pronouncements from established authorities has moved into retirement homes. The younger generation is far more interested in its own opinion, which it shares in virtual communities like Yelp, MySpace, and Facebook. It values transparency among peers (if not in the comments sections of blogs), and its members are strung out on the 24-hour news cycle, which has them addicted to the latest 140-character information bomb from Twitter.
We do not live in a Zagat World anymore.
This epiphany came to me via the 2010 edition of the Washington D.C./Baltimore Zagat guide. There, on page 10, are the 40 highest-rated restaurants in the D.C. area in terms of food. At the top, for the second year in a row, is Makoto, a 25-seat Japanese restaurant in Palisades that prepares a pristine, multicourse omakase menu based on the seasons and the chef’s whims. The place earned 29 out of 30 points for food from Zagat raters, just barely beating out the Inn at Little Washington, which is rather impressive given the latter’s bona fides. In 1994, the International Herald Tribune named the Inn one of the 10 best restaurants in the world, and it has remained a darling of critics. Celebrities and politicians hop in helicopters to dine at the Inn at Little Washington. Washingtonians consult their GPS to figure out how to find Makoto.
You might be wondering how a niche restaurant like Makoto, with such a minuscule seating capacity, could generate enough votes to win the top spot two years running in the D.C. Zagat guide. There’s an easy answer: It didn’t. Makoto won the 2009 Zagat survey for food. The ratings in the 2010 book merely repeat those from last year’s survey, although the casual reader would be hard-pressed to know this important fact.
The only way I learned about the duplicate ratings was through Michael Birchenall, editor and publisher of the local trade magazine Foodservice Monthly. After reading a blog item that I wrote about Makoto’s strange stranglehold on Zagat, Birchenall combed through his old guides and discovered an interesting trend: Those restaurants that topped the Zagat ratings in the odd years were the same ones that topped them in the even years. A spokesperson for Zagat confirmed his findings.
“As Michael Birchenall pointed out to you, we compile new survey results and prepare a new guide for Washington, DC/Baltimore every other year,” e-mailed Tiffany Barbalato, director of communications for Zagat. “This is why the winning restaurants and top lists you refer to in the 2010 guide are the same as last year’s.”
Barbalato, in the same e-mail, alerted me to this line in the latest guide: “This 2010 Washington, DC/Baltimore Restaurants Survey is an update reflecting significant developments since our last Survey was published.” (Those significant developments, incidentally, are mostly the addition of new, unrated restaurants with an editor-written description.) Barbalato offered up this lone, coyly worded sentence as evidence that Zagat doesn’t try to dupe its customers about the duplicate nature of the even-year guides.
I had my doubts that this anemic sentence was pulling its weight, so I called a few restaurateurs and asked them if they knew about Zagat’s duplicate ratings. “I did not know that,” says Jeff Black, 46, the owner of four restaurants, including BlackSalt and Addie’s, who’s been working in the hospitality business since age 13. “That’s kind of lame.”
“Oh, really?” says Barton Seaver, chef at the new Blue Ridge in Glover Park. “OK, that’s a dinosaur.…We’re living in an era where Todd Kliman [of the Washingtonian] reviews a restaurant as it happens” via Twitter.
Their alarm at this news is understandable. Many restaurants have shorter life spans than the average prime-time program, which means that these places grow, mature, and gray quickly. A Zagat rating based on a year-old survey—or older—is the equivalent of judging this season’s Mad Men by the episodes of the previous season. Take, for example, the rating in the 2010 Zagat guide for Black’s Bar & Kitchen. Jeff Black’s Bethesda operation scores a respectable 24 for food, but that rating is based on a survey likely tabulated when Mallory Buford was executive chef in mid-2009. Black’s is now on its third different executive chef since Buford left. “Things do change [at restaurants],” Black says, “and they change quickly.”
Makoto may be one of the few exceptions to that rule. Time doesn’t stand still here—but it actually seems to expand, as if you get two minutes for every 60 ticks around the clock. Part of the sensation can be traced to a pair of unyielding policies at Makoto: You must take off your shoes in the anteroom and don slippers, and you must silence your cell phones. These house customs leave your toes swaddled in pillowy comfort and leave you to contemplate the finer things about the restaurant experience: your food, your dining companions, and your deepest, most neurotic thoughts.
Depending on the quality of the latter two, it’s often best to focus on the food. Yoshi Itoh, chef and co-owner of Makoto, has developed a disciplined kitchen that can quickly produce a small banquet of plates, some so good you’ll wonder why there’s not a line snaking around this restaurant every night. Deep-fried soft-shell crabs breaded with pebble-sized crumbles of rice cracker. Strips of medium-rare tenderloin so rich and tender they practically slide down your throat with the soy-based sauce. Wasabi-smeared slivers of fresh fish perched atop the fluffiest sushi rice you’ve ever seen, each piece of nigiri expertly balancing its heat with its sweet.
And yet for all its devotion to the fine art and technique of Japanese cooking, Makoto is not one point away from perfection, at least not according to my own internal guide. The tuna served as part of my sashimi course during a recent visit was mealy and gummy, while the flounder was flavorless. The dessert of shaved ice, flavored with fruit and Grand Marnier, was refreshing enough but, frankly, has worn out its welcome after countless appearances on Makoto’s omakase menu. Then there’s the squadron of female servers, who balance the gentility of a geisha girl with the fastidiousness of an English nanny. I can’t tell if they want to sit with me or tell me to clean up my room.
So how does a place like Makoto reach the pinnacle of the Zagat food ratings? If you research the published literature on Zagat, you’ll come across stories of restaurateurs trying to game the voting system. They’ll blast e-mails to the diners in their database, reminding them to cast their votes before the Zagat survey period ends. Their message is implied but clear: Stuff the Zagat ballot box for their restaurant. Some restaurateurs might even offer discounts, or other gifts, to those diners who cast ballots, despite the fact that this kind of tit-for-tat vote solicitation can get you banned from the Zagat book.
Jeffrey Buben, owner of Bistro Bis and Vidalia, knows for certain that some restaurateurs try to outsmart the Zagat system. He’s one of them (though without the promise of any kitchen bribe). In the past, he’s placed little promotional tents on the tables or slipped cards into the checks, asking diners to vote in the next Zagat survey. His goal is to land a Top 5 ranking in Zagat’s food category. He doesn’t apologize for it, either. “That’s what grass-roots PR is all about,” Buben says. He leaves it up to the Zagat data-crunchers to weed out the questionable ballots, which they claim they can do.
Makoto has accomplished exactly what Buben was aiming for. The restaurant opened either in 1992 or 1993 (neither Makoto manager Michiko Lecuyer nor anyone at the restaurant could remember), and for its first few years, it never even appeared in the Zagat guide. But then in 1997, it suddenly scored 28 out of 30 points for food, tying it for second place with L’Auberge Chez Francois in the entire D.C.market. Makoto has never slipped lower than third place since then, a run of more than 12 years.
Lecuyer credits the kitchen for Makoto’s long ride atop the charts. The chefs select only the freshest seasonal ingredients, from fish to vegetables, no matter what the price, she says. The cooks also prepare the omakase plates with a nod to authenticity. “We serve the food we want to eat,” Lecuyer says, “the way we want to eat it.…Our customers get it and understand it. That’s why we’ve been highly listed.”
Now, if there’s one question you don’t want to ask the owner or manager of an elegant Japanese restaurant like Makoto, it’s this: Do you beg or cheat for Zagat votes?
“We don’t do that,” Lecuyer says, noting that servers aren’t allowed to talk to customers. “We don’t know who is doing the survey and is not doing the survey. We don’t know.”
Such ignorance may be a blessing to Makoto, but it’s a bane for diners who want to know how legitimate Zagat’s ratings are. Team Zagat in New York won’t answer any questions about its survey methodology, and Tim and Nina Zagat turned down my interview request. It’s more of the same stonewalling that has worked for 30 years for the Zagats and their guides, which now cover subjects ranging from dining to golf in more than 100 countries. The company’s attitude forces you to make a snap decision: You either trust it or you lump it.
A lot of people are opting for the latter. According to a New York Post article from earlier this month, not only are Zagat sales “down dramatically,” but the company moved too slowly online, “allowing Yelp and others to dominate the market.” Zagat, the paper wrote, laid off about 16 people in May. The founders themselves seemed to see the writing on the wall a year earlier: The Zagats tried to sell their company for a reported $200 million last year but couldn’t find any buyers in that price range and pulled Zagat off the auction block.
It’s easy to see why people choose Yelp, Chowhound, Urban Spoon, and OpenTable over Zagat, both the guide and the online site. The obsolete-before-you-buy-it Zagat book is $14.95 a shot, and Zagat.com charges nearly $25 a year to access its complete site, including those ratings that can be seriously outdated. Plus, the mere act of creating an account with Zagat.com requires that you provide the kind of personal information—mailing address, telephone, birth year—that other sites have decided to forgo.
By contrast, Yelp, Chowhound, Urban Spoon, and other sites are free, and most already have established communities where members interact with each other on particular topics and restaurants. Even better for diners looking for recommendations on Yelp or OpenTable, they can see exactly how many people have commented on a particular restaurant—and how those reviews have been averaged into an overall rating. Even the minimal transparency on these sites makes Zagat seem like Stalinist Russia.
“Zagat is not a primary source [for information] anymore,” says Dean Gold, the chef and owner of Dino in Cleveland Park, which scores a decent 21 rating for food from Zagat. Adds Gold, a college-trained statistician: “Zagat, of all the major sources, probably has the lowest levels of reliability” because of its self-selected survey base, which provides little to no information on the people who actually cast ballots.
At this point, OpenTable may be the most reliable of the sites that aggregate restaurant ratings. Site administrators send review surveys only to those diners who have honored their OpenTable reservation, and the diners have approximately 30 days to fill out the forms. This process guarantees two things, says Ann Shepherd, vice president of marketing for OpenTable: 1) that every review is actually based on a meal eaten at the restaurant; 2) that the meal was eaten recently, while the memory of it is still fresh.
Those are two promises that you will never hear from Zagat, a guide that looks destined to follow so many other print publications into oblivion.
Zagat’s 2010 Makoto blurb shows why the guides are “refrigerator-magnet poetry,” “dubious,” good only for a “lonely traveler…searching for a good place to sup before masturbating himself to sleep.”
Grade Creepiness: As first pointed out in SmartMoney magazine in 2007, Zagat food grades have spiked dramatically over the years for restaurants in the New York guide. Same goes for the D.C. area: In the 1992 guide, only 13 restaurants earned grades of 25 points or higher (out of a possible 30). In 2010, more than 60 restaurants topped the 25-point mark. Even more startling, 72 percent of D.C. restaurants with actual ratings in the latest guide earned a grade of 20 or higher, which means that nearly three-quarters of our eateries are “very good” or better. No one sucks here anymore.
The Method, Man: Readers have no idea how many votes are required before the results are considered statistically relevant to merit a Zagat grade (Zagat’s Barbalato told me that Makoto edged Inn at Little Washington, 28.9024 to 28.8495, but would not say how many votes were cast). What’s more, they have no idea if the grade represents the votes of the restaurateur’s spouse and 100 close friends or a statistically sound sample of D.C. area diners. All they know is this: The voters are self-selected, which is a pool almost guaranteed to skew results. Readers don’t even know if voters actually ate the restaurants in question. OpenTable, by contrast, posts reviews only from diners who have honored their online reservation.
Cut-and-Paste Prose: Zagat editors take a ransom-note approach to writing descriptions of the restaurants in their guides. Compare that to consumer-oriented sites, such as Yelp or DonRockwell, where amateur critics can relate their entire dining experiences without fear that an editor will place a dis about “uncomfortable seating” right next to some yahoo’s Pollyanna piffle about a “wonderful experience.” Zagat is refrigerator-magnet poetry at a time when people want the Library of Congress at their fingertips.
You’ve Been Duped: For those who purchased the 2010 edition of the Washington D.C./Baltimore Zagat guide, the Makoto entry might seem familiar. For good reason. It’s the exact same entry as last year’s. The exact same awkwardly phrased copy. The exact same dubious grades. And yet you’d be hard-pressed to learn from the introduction that the 2010 guide is merely an update. Here’s the truth: The even-year Zagat guides are based on the surveys from the previous year. Do you know how much a restaurant can change in two years, let alone two months? Would you trust a review from a year ago on Yelp?
Solitary Confinement: Zagat bases your estimated check on a single dinner and drink, plus tip, which is rather symbolic. Social network sites want to create a community around a common interest in food, which explains not only Myspace Local’s recent move into online restaurant reviews but also local restaurateurs’ embrace of Facebook and its ability to connect supporters. Zagat, by contrast, still conjures up images of the lonely traveler, that burgundy guide tucked into his back pocket, searching for a good place to sup before masturbating himself to sleep.
Caught in a Binder: Like newspapers and magazines, Zagat is dependent on print, where presumably the company still earns most of its revenues, despite the fact that its most timely and user-friendly information (menus, maps, and recent reviews) is found online. Zagat withholds survey ratings from its free Web pages in hopes that you’ll plunk down $24.95 a year to find out what the voting public, in whatever numbers, thought of places like Makoto over a year ago. It’s a hopeless online business plan, but it’s probably a better deal than the $14.95 you pay for the actual paperback guide.