Our recipes and stories, delivered.

April 30, 2019
An Internet Recipe Watchdog Speaks
IMG_1885

A blogger is working her way through the Internet’s baking recipes, one detailed spreadsheet at a time.

When Erika Kwee wants to find the best recipe for chocolate cake, her process takes a bit longer than the average Google search. Kwee, who runs the blog The Pancake Princess, has gained a following (over 25,000 people on Instagram alone) from her bake-offs, where she tests recipes from across the Internet to determine the most reliable. Any online recipe is eligible for inclusion in one of Kwee’s bake-offs—be it from a digital or print publisher, cookbook reprint, food personality, restaurant, or blogger—and all are judged using a blind testing method.

Kwee’s “data-driven baking” began quite accidentally. Though she and her friend Skyler Johnson dreamed up the idea of using data to find the best chocolate-chip cookie recipe years prior, they didn’t actually follow through until 2017. Johnson was visiting Kwee’s home in Houston, Texas, when Hurricane Harvey hit, and suddenly they were stranded.

“We were so bored by the end of the week that we were like, ‘Could we just do the cookie thing?’” Kwee tells me.

And so they did. They baked over 200 cookies from 12 recipes (now she tries to limit the bake-offs to nine recipes) and had 30 friends taste and rate the cookies. Kwee photographed the results, and Johnson, who has a background in data-visualization design, translated their findings using the software program Tableau—making it easy for readers to compare techniques and ingredients with the resulting cookie products. They crowned Cook’s Illustrated’s Perfect Chocolate Chip Cookies the winner. After the cookie post’s success on her blog and Instagram feed, Kwee, who works in product management at HP by day, was inspired to publish more bake-offs. Now she has 14 under her belt.

As it turns out, a print magazine winning one of her bake-offs became a bit of an anomaly. Kwee’s research shows that recipes published on self-published blogs typically fare a bit better among her tasters than those by recipe developers working for publishers or companies.

Bon Appétit has been up there. Cook’s Illustrated and King Arthur Flour have all done well in certain tests, but some bloggers will just blow everyone else out of the water,” Kwee says, noting that Sarah Kieffer’s cinnamon roll recipe on The Vanilla Bean Blog (which was up against America’s Test Kitchen and King Arthur) was by far the favorite in a recent bake-off. Another blog, Tastes of Lizzy T, won second place. Bloggers have also won or come in second in Kwee’s pumpkin pie, sprinkle cake, brownie, and banana bread contests.

Kwee has a hunch about why blogger recipes do well, even when they’re up against recipes that come from robust test kitchens: “They’re only beholden to themselves. If they don’t like a recipe, they don’t have to post it until it’s ready.”

A few bloggers I spoke to corroborated this logic. “If [the recipe is] for my blog, and there is zero deadline, I may work on it for weeks, sometimes months, until I get it just right,” says Rebecca Firth, who runs the blog Displaced Housewife.

“I like to have enough time to, at the very least, test the recipe twice,” says Jessie Sheehan, who runs the blog Jessie Sheehan Bakes. “And if the recipe does not work, or is not perfect the first time, then I will keep remaking it until it is—which might be anywhere from four to eight times.”

On the other hand, several recipe developers I spoke with who work for digital and print media publishers (all of whom requested anonymity) cited frustration with the short amount of time they’re given to produce content. An associate food editor for a popular magazine and its website said she’s typically given just one or two days to develop a recipe.

Another food editor, who works in the test kitchen at a website, develops 10 to 15 recipes per week, testing the recipes once or twice before they’re published. Sometimes she’ll get emails from readers about a recipe flopping, and in those cases the kitchen will retest and edit the online recipe.

Kwee starts each bakeoff by Googling a phrase like “best chocolate cake,” searching her favorite blogs, and collecting reader suggestions. She then inputs the recipes on a spreadsheet—she calls this her “low-tech version of web scraping”—to compare the ingredients, and will cut any recipe that seems too similar to another, which Kwee says happens often: “Many of them are just slight adaptations of each other.”

The chocolate cake recipes Kwee scraped proved to be more unique than some of her other bake-off subjects: She found recipes that used oil, sour cream, butter, and (to her horror) mayonnaise, as well as combinations of two fats. Because of these discrepancies, Kwee settled on 12 recipes to test instead of her more typical nine.

She baked all the cakes on the same day to ensure equal freshness, used a control frosting, and brought in testers (there can be anywhere from nine to 50; for this one, she had 31). Each tester filled out a survey with a 1-10 rating system on flavor and texture and questions like “How would you describe the moisture level?” and “Would you want to eat this again?” Kwee photographed the cakes and transferred data from the surveys into graph form, which she used to declare a winner. For chocolate cake, that was Ina Garten’s vegetable-oil-and-buttermilk-based recipe.

Ultimately, Kwee’s project doesn’t bash recipes. Every bake-off includes detailed descriptions of each recipe’s merits and a superlative, like “the best classic,” “the best flavor for the least amount of work,” “the best when you want to impress ‘foodie’ friends.”

She doesn’t claim to be the be-all, end-all authority on the best recipes, as she knows the nature of a taste test is a subjective, rather than airtight scientific, process. Most of Kwee’s testers are people in their 20s and 30s who care about food. She noted an example of a big data shift when she added the results of her brother’s college friends to the chocolate-chip cookie bake-off data—while Kwee’s friends preferred cookies made with browned butter and sprinkled with salt, her brother and his friends’ palates were totally different, rating a no-frills cookie (that Kwee politely deemed “inoffensive”) their favorite. “As much as I try to make this a very rigorous data experiment, I realize that it’s such a small data sample.”

Even for a relatively small data sample, the bake-offs are a lot of work for a part-time blogger.  Kwee mentioned it was a bit discouraging to see that her posts weren’t gaining much traction after the first few bake-offs. “But I somehow felt like I couldn’t stop,” she says. Eventually, a repost from King Arthur Flour led a larger audience to her project. This engaged following and her genuine curiosity about recipes continue to motivate Kwee.

Ultimately, Kwee hopes her data-based baking simply helps readers make more informed decisions about recipes. She’ll often get notes from readers who’ve used her posts to find their new go-to recipe, or who were making one recipe for years until she inspired them to try something new.

“At this point, I don’t have any serious end goals in mind,” Kwee says, naming biscuits, blueberry muffins, apple pie, and snickerdoodles as a few upcoming projects. “Based on all the requests I’ve gotten for future bake-offs, this project could probably go on forever.”

Rebecca Firkser

Rebecca Firkser is a New York City-based food writer and cook. Most recently the Culinary Editor at Extra Crispy, Rebecca’s byline has appeared in number of publications, among them Food52, Tasting Table, and Healthyish by Bon Appetit.