Bounces are not being handled

Hello,
I have successfully installed PHPList and configured it with Amazon AWS.
I’ve created a test campaign with 3 real emails and one fake impossibly long email address to test bounces.
I have set up an email address in my cPanel to handle bounces and I’ve set the config.php to handle bounces but when I go to System > Manage Bounces > Process Bounces I get this message:

0 bounces to fetch from the mailbox
Identifying consecutive bounces
Nothing to do
total of 0 subscribers processed                            

I also looked at the mailbox in cPanel and it appears there is no messages there.

Do I have to setup something somewhere (other than config.php) to make sure bounces are actually sent to that email?

Here are my config.php Bounce settings:

// Message envelope. This is the email that system messages come from
// it is useful to make this one where you can process the bounces on
// you will probably get a X-Authentication-Warning in your message
// when using this with sendmail
// NOTE: this is *very* different from the From: line in a message
// to use this feature, uncomment the following line, and change the email address
// to some existing account on your system
// requires PHP version > "4.0.5" and "4.3.1+" without safe_mode
$message_envelope = 'xxx';

// Handling bounces. Check README.bounces for more info
// This can be 'pop' or 'mbox'
$bounce_protocol = 'pop';

// set this to 0, if you set up a cron to download bounces regularly by using the
// commandline option. If this is 0, users cannot run the page from the web
// frontend. Read README.commandline to find out how to set it up on the
// commandline
define('MANUALLY_PROCESS_BOUNCES', 1);

// when the protocol is pop, specify these three
$bounce_mailbox_host = 'xxx';
$bounce_mailbox_user = 'xxx';
$bounce_mailbox_password = 'xxx';

// the "port" is the remote port of the connection to retrieve the emails
// the default should be fine but if it doesn't work, you can try the second
// one. To do that, add a # before the first line and take off the one before the
// second line
#$bounce_mailbox_port = '110/pop3/notls';

// it's getting more common to have secure connections, in which case you probably want to use
$bounce_mailbox_port = "995/pop3/ssl/novalidate-cert";

// when the protocol is mbox specify this one
// it needs to be a local file in mbox format, accessible to your webserver user
$bounce_mailbox = '/var/spool/mail/listbounces';

// set this to 0 if you want to keep your messages in the mailbox. this is potentially
// a problem, because bounces will be counted multiple times, so only do this if you are
// testing things.
$bounce_mailbox_purge = 1;

// set this to 0 if you want to keep unprocessed messages in the mailbox. Unprocessed
// messages are messages that could not be matched with a user in the system
// messages are still downloaded into phpList, so it is safe to delete them from
// the mailbox and view them in phpList
$bounce_mailbox_purge_unprocessed = 1;

// how many bounces in a row need to have occurred for a user to be marked unconfirmed
$bounce_unsubscribe_threshold = 5;

// Set to 0 to received by mail bounce deletions in the advanced bounce processing report
define('REPORT_DELETED_BOUNCES', 0);

I am having a similar issue and I don’t know if it is phplist or Amazon SES. I’ve been using phplist with SES for over six months and have never gotten a message bounced back. I know that cannot be correct. I’ve enabled e-mail forwarding, but even when I send a phplist test message to bounce@simulator.amazonses.com nothing ever shows up in the mailboxes I’ve configured to receive e-mail bounce forwarding. I’ve been trying for several months now to figure out what the problem is.

@marcnyc Have you added the $message_envelope email address as a verified address in your SES console?

https://docs.aws.amazon.com/ses/latest/DeveloperGuide/monitor-sending-activity-using-notifications-email.html

That indeed seems to have been the problem… To bad there isn’t a guide that details all of these extra steps one must take… thanks so much for helping…

Do you know if on SES you should have any of the following settings set differently than the default (which is this)?

Bounce Notifications SNS Topic:

none

Include Original Headers:

disabled

Complaint Notifications SNS Topic:

none

Include Original Headers:

disabled

Delivery Notifications SNS Topic:

none

Include Original Headers:

disabled

I’m sorry to report the Bounces still don’t work.
I’ve tried adding another fake and impossibly long email address as a subscriber, after the campaign is sent, if I go to PROCESS BOUNCES it does say there was 1 email in the mailbox and that the mailbox was purged (so the email did bounce and it went from Amazon to my bounce pop3 account) but after this the PHP LIST Stats don’t show any bounces. It says all emails were delivered.
What am I missing so these bounces are actually displayed as such in PHPList so I know what to do with them?
Thank you

Well, I had set up the envelope correctly, but apparently over time I had forgotten I had! So lately I’ve been looking at the wrong e-mail for the bounces! I have determined now that Amazon is forwarding the bounce notifications correctly, but phplist has not been seeing them in the box, so I will have to look into that now.

@dschaller @marcnyc If phplist is retrieving from the mailbox then the bounces may be appearing on the Unidentified tab when you view bounces. That means phplist cannot identify the subscriber or the campaign.

Update
Using the bounce@simulator.amazon.ses address phplist did identify the bounce

I have checked that and I don’t see anything in System > Manage Bounces > View Bounces > Unidentified

Also under Subscribers > Search Subscribers it says:

udhnoisdfnpasufnahgdjfa@kjhsdfkgfdkghsdflighsdglhsgsdfuh.com
msgs: 1

But it doesn’t say bncs: 1

I also tried adding the bounce@simulator.amazon.ses address to my list and when I processed the bounces I got this:

1 bounces to fetch from the mailbox
Please do not interrupt this process
Closing mailbox, and purging messages
Identifying consecutive bounces
total of 1 subscribers processed

But then again I don’t see any bounces in the VIEW BOUNCES (Processed or Unidentified) or SEARCH SUBSCRIBERS pages…

What I did notice thought is that in the VIEW BOUNCES page there seem to be more bounces that are labeled as SYSTEM MESSAGE and are assigned to the Subscriber that is also my FROM address… That is strange…

@marcnyc Have you examined the system emails to see whether they match your expected bounce? Perhaps the bounce was associated with the From address instead of the message_envelope address.

I had deleted them to be able to know what is new after every test. Now I just ran another campaign and there are no such system emails and no bounces to be seen anyway… very frustrating… still in the subscribers page it shows all emails were delivered and none were bounced… i’m at a loss

If the manual bounce process flag is set to 1 in config, does that mean that even if a cron job is setup phplist will not execute? I set it to 1 thinking that then both would be in place at once-- namely, it would process bounces daily, but I could also manually force bounce processing in-between cron jobs. But is that setting mutually exclusive?

@dschaller

define('MANUALLY_PROCESS_BOUNCES', 1);

This allows the process to be run from both the admin interface and a cron job.

I have that set to 1 but it still doesn’t process the bounces correctly

Any other ideas on how this can be addressed / fixed?

Although you have the protocol specified as POP

$bounce_protocol = 'pop';

You have an mbox setting uncommented further down:

$bounce_mailbox = '/var/spool/mail/listbounces';

In my bounce settings, the $bounce_mailbox line is commented out since I’m using POP. Could it be a problem that you have both POP and mbox settings active?

I am cautiously optimistic that my bounce processing is working now! I discovered that my hosting provider had changed the POP server settings on my mailboxes without any notification [hits forehead] so phplist has been looking for the wrong server. I never noticed because I’ve been checking those boxes via web interface. Anyway, I tested with bounce@simulator.amazonses.com and bounce processing appeared to work in phplist now as advertised. (Have not checked it using the cron job yet, but there’s no reason I shouldn’t get the same result there, right?)

Thank you… I will try this suggestion…

The whole Bounce managment thing could use some improvements I think… I mean I think all the Bounce related things should be in the same place…
now there is:

  • SYSTEM > MANAGE BOUNCES (which also has List of Bounce Rules)
    and yet then there is…
  • CONFIG > BOUNCE RULES
    …but also there is this weirdly named menu called:
  • STATISTICS > msgbounces (which I am unclear as to what it does)
    …and at the same time there is:
  • SUBSCRIBERS > View Bounces Per List (which seems the same as the previous)
    …and yet if you finally wanna do something about bounces you have to go to:
  • SUBSCRIBERS > MANAGE SUBSCRIBERS > RECONCILE SUBSCRIBERS
    …which is really a bit too hidden!!!

Can’t we just have all of these 5 things under ONE menu item called BOUNCE MANAGMENT?