Headline Testing: Mailchimp

I’m looking for a simple, cheap solution to test headlines on a website that generates less than 600 views a month. I’ve identified a few companies that offer this service; here, I’ll reprint their replies to my questions. Below: Mailchimp.

From Meads via email on March 9, 2020:

We recommend you have at least 5,000 subscribed contacts to get the most useful data from your subject-line test.

Headline Testing: VMO

I’m looking for a simple, cheap solution to test headlines on a website that generates less than 600 views a month. I’ve identified a few companies that offer this service; here, I’ll reprint their replies to my questions. Below: VMO.

From Tushar Sureka via email on March 9, 2020:

VWO works on the latest Bayesian Stats model to declare faster and accurate results.

We need a minimum of 1,500 visitors per test to declare a statistically significant result.

Please go through this article for some recommendations on how to do testing on low-traffic websites.

Headline Testing: Thrive Themes

I’m looking for a simple, cheap solution to test headlines on a website that generates less than 600 views a month. I’ve identified a few companies that offer this service; here, I’ll reprint their replies to my questions. Below: Thrive Themes.

From Mario via email on March 7, 2020:

As long as there is a steady influx of visitors, some trends should still be visible in order to make some decisions regarding your conversion approach.

600 views/month is not a huge number, but smaller sites have been able to get some conclusions from the test that they run on even fewer than 20 visitors/day.

That said, the longer the time you use to test something out, the more likely the results will be more accurate.

I replied, the same day, as follows:

1. How many visits do you recommend in order to get a result that’s statistically significant?

2. How many variations do you recommend testing?

Headline Testing: Nelio A/B Testing

I’m looking for a simple, cheap solution to test headlines on a website that generates less than 600 views a month. I’ve identified a few companies that offer this service; here, I’ll reprint their replies to my questions. Below: Nelio A/B Testing.

From David Aguilera via email on March 7, 2020:

Indeed, A/B testing works best on websites that have mid to high traffic. When run on low-traffic sites, it’s difficult to get meaningful results because there simply isn’t enough data.

If you have a low-traffic site, we recommend you run radical tests with only two variants (A and B) in those pages that get the most traffic, then track conversion actions that are in early stages of your conversion funnel.

So, for example, instead of testing different color buttons or a different copy, create a completely different landing page. The more radical the changes, the more likely it’ll be to see any statistically significant differences between variants A and B.

U.K. Audiences Dislike What U.S. Audiences Love

Much of what works so well in American BuzzFeed posts—nostalgia, sentimentality, uplift—doesn’t work with British audiences. “That stuff just completely bombs in the U.K.,” BuzzFeed U.K. editor Luke Lewis said. “Nobody wants to be uplifted, particularly. But if you can make the articles funny, and a bit needling and a bit satirical, they do well.”

London Calling: A Look at BuzzFeed’s British Invasion

A Sucker Is Optimized Every Minute

“For years, search-engine optimization, or S.E.O., has turned web pages into Googlebait. These days, optimizers of squeeze pages, drawing lessons as much from the labcoats at Optimizely as from the big daddies at Google, recommend creating a three-to-10 minute video that’s introduced by a ‘magnetic headline’ (‘Find the Perfect Lampshade for Any Lamp’) and quickly chase it with an ‘information gap’ like ‘You’re Not Going to Believe the Trick I Use While Lampshade Shopping.’ (Article of faith among optimizers: humans find information gaps intolerable and will move heaven and earth to close them.) Next you get specific: ‘Click the play button to see me do my lampshade trick!’—after which the video unspools, only to stall at the midpoint with a virtual tollbooth. You can’t go on unless you hand over an email address. Presto.”

—Virginia Heffernan

Why Your Homepage Should Be Different for Different People

Most companies have one homepage, which they show to every visitor, regardless of his purchasing power. This simplifies things, but it also leaves opportunity on the table. What’s persuasive to the CEO of a Fortune 500 firm will not resonate with your local pizzeria.

Here’s HubSpot default homepage. It displays three different case studies, from three different-size companies:

If, however, HubSpot identifies a visitor as coming from an enterprise-size company, it displays the logos and case studies of its biggest clients:

The result of this personalization: a 42% jump in clicks on HubSpot’s calls to action.

How Bloomberg Businessweek A/B Tests Its Covers

Cover Trail: Tim Cook’s Apple

How Bloomberg Businessweek A/B Tests Its Covers

Cover Trail: Can We Talk About the Weather Channel?

Pop Quiz: Which Facebook Post Did the Best?

A. Close-up of Justice Ginsberg, pretty P.O.ed, asking everyone to Join the Dissent ad

B. Pretty sweet nostalgic I Love Lucy ad

C. The full Supreme Court, being called out for ruling against women ad

Answer here.

Even Upworthy’s CMS Is Edgy

“Curators load potential headlines and thumbnail images into a testing system, which shows each option to a small sample of the site’s visitors, tracking their actions—did they click it, did they share it? The system used to return detailed numerical feedback on each option, but it was decided that hard numbers over-influenced the curators; now it tags options with things like ‘bestish’ and ‘very likely worse.’”

Watching Team Upworthy Work Is Enough to Make You a Cynic. Or Lose Your Cynicism. Or Both. Or Neither

As You Title-Test, Don’t Forget to Picture-Test

Upworthy Goes Viral by Optimizing Optimism

If You’re Not Scheduling Your Posts, You’re Doing It Wrong

Obviously, you can’t schedule everything. Nor would you want to. But you ought to be assessing your analytics for your optimal times and days. As Megan Garber reports in her must-read profile of Imgur (pronounced “IM-uh-jur”),

Imgur, like many sites, seems to owe the rhythms of its traffic patterns to the rhythms of the office workday. The site’s peak traffic tends to come at 2 PM Pacific time, Matt Strader, Imgur’s COO, told me—the closing of the work day on the East Coast, and the tail-end of lunch breaks on the West. (Monday is usually the biggest day for Imgur traffic, likely the result of people settling back into the work week, and weekends are the lowest.)

Your Gut Is Gullible

Last year, a senior member of Obama’s campaign email team dropped this bombshell: despite the team’s expertise, no member could reliably predict which message would generate the most money. “We basically found our guts were worthless,” he admitted.

Earlier this year, Adam Mordecai, Upworthy’s editor at large, divulged the same deficiency. “I’m our highest performing curator,” he wrote. “I get 20-30% of our traffic every month. And every time I predict something will go viral, or headline X will win, I’m usually horribly wrong, usually about 97% of the time.”

The lesson: test, test, and test some more.

Addendum (12/23/2013): Or, to put it another way: “Unless you harness the magical powers of a unicorn horn, you will never know how to make all your stuff go totally viral.”

Why You Shouldn’t Trust Your Gut

Which call to action do you think will perform the best?

• Sign Up
• Learn More
• Join Us Now
• Sign Up Now

As the Obama campaign learned, the difference is worth tens of millions of dollars.

Of course, predicting a winner isn’t intuitive. With remarkable candor, one Obama campaign spammer emailer put it this way: “We basically found our guts were worthless.”

Really? Surely, “join us now” will outperform “sign up.”

Nope. As former Obama operative turned Optimizely CEO, Dan Siroker, documents in the above video, “Learn more” carried the day.

Lesson: even the experts are still experimenting. Or, to paraphrase Reagan, Trust, but test.

25 Headlines

25: that’s how many headlines Upworthy requires its writers to draft for every article. Wow!

What’s more, Upworthy then tests the headlines to see which ones spur the most clicks. The payoff: a click-baitish headline can drive upwards of 500% more traffic than a boring one.

Indeed, here’s the headline-heard-round-the-net (not from Upworthy but from Forbes):

* How Companies Learn Your Secrets

* How Target Figured Out a Teen Girl Was Pregnant Before Her Father Did

Which would you click on?

If you analogize a headline to a tweet or status update, the same principle applies: the more swings you take, the greater your chance of hitting a home run.

So, don’t just go with the first headline that comes to mind. Develop multiple options. Play around with questions, declarations, listicles. If you have the time, maybe poll your colleagues to see if a consensus emerges.

The more we can subject our suppositions to data, the more “Hey”-like results we’ll experience.

“Our Guts Were Worthless”

“‘We basically found our guts were worthless,’ observed a senior member of the [Obama] campaign’s email team, on the fact that nobody on the team could reliably predict which emails would perform best. The campaign committed itself to a relentless regimen of experimentation, test and test again. The team ‘regularly tested’ as many as 18 variations on subject line and email copy.”

à Invest in Digital Marketing to Control Your Destiny