A/B Testing Email Subject and Body

ℹ️ Support
,
1

Greetings phpListians!

Is there a way to send two versions of a campaign to a list and see afterword which version has a higher open rate and/or click-through conversion rate?

I can imagine a few ways to A/B test campaigns but I’m not sure which to try:

  1. Send a ‘url’ type campaign and handle the alternate versions and click tracking with an external script that generates the message.
    • I’m not sure if phplist would re-load the url for each subscriber and therefor it may not work at all (I haven’t tested that). I’d like to use this method, though disadvantages would include not being able to test alternate subject lines and test results not being available within phpList stats.
  2. Manually divide subscribers into two lists, send two different campaigns, analyze the results and then recombine the lists.
    • This seems like it would work but would be pretty labour intensive and prone to user errors with all the editing of the lists.
  3. Send a campaign to half the list using the segment plugin, copy and edit the campaign, send to the other portion of the list using segment plugin.
    • One challenge with this (and option 2) would be dividing the list in a way that was random enough that it would not skew the results.

Is there an existing option or plugin for this task that I’ve missed?

Thanks for any suggestions!

2

It’s fairly easy to divide all subsribers from a subscriberlist.
You export them as a csv, filter out the blacklisted users and divide the csv in two.
Kind regards,
Peter

3

I would love to see A/B testing supported directly, mainly because I’m too lazy to do it manually. :grin:

I would use a single list with two campaigns, and a (hidden) user attribute (perhaps called “ABTestGroup”) with the Segment plugin. As @Peter-sdl suggested, I would download the list subscribers (confirmed and not blacklisted) and open it as a spreadsheet. Then I would delete all columns except the email, add a new column for the user attribute, assign “A” to one half and “B” to the other half, export to CSV, then import that (not selecting a list).

This will assign people to either the A or B group, so then you can send your A campaign to the A group and your B campaign to the B group. Results analysis will be manual, of course, since there’s nothing that compares the results for two campaigns.

There’s not much value in just one iteration, though, so it might be better to assign more groups up front. Maybe 10% get A, 10% get B, 10% get C, etc. First round is A & B, next round is the round-1 winner plus C, next round is the round-2 winner plus D, etc. The first two campaign versions would be written up front, while the third and successive versions would probably be written based on the results from the prior round(s).

One can see how my laziness is an obstacle here, lol.

1 Like