So, I’ve had our IS person set up a Gmail account for me for dealing with phpList bounces.
And, per a different support thread I installed and enabled the ImapPlugin and adjusted the config.php file the way I thought it needed to be adjusted:
Set “$message_envelope” to the email address that our IS person set up and uncommented it.
Set the various “$bounce_mailbox_X” values to match up with what Google said to use for IMAP stuff: “imap.gmail.com” for host, and the Google Account’s user/password for those settings.
Set “$bounce_mailbox_port” to “993/imap/ssl”
But when I go to the phpList back-end and choose Manage Bounces → Process Bounces it thinks and thinks and eventually comes up with a screen that says:
Cannot create POP3 connection to imap.gmail.com:
Identifying consecutive bounces
Nothing to do
total of 0 subscribers processed
So I’m thinking that I’ve missed a setting somewhere or something, because it should be working, right?
Hrmmm…so I deleted the IMAP plugin and installed/enabled the IMAP2 plugin and left everything in the config.php file the same and now when I go to Manage Bounces → Process Bounces, it very quickly comes up with the following:
Cannot create POP3 connection to imap.gmail.com: Can not authenticate to IMAP server: [AUTHENTICATIONFAILED] Invalid credentials (Failure)
Identifying consecutive bounces
Nothing to do
total of 0 subscribers processed
I double-checked that I have the Gmail account’s username/password correct in the config.php, and that all checks out, so I have no idea what might be going on…
Crap, so the way they have their Gmail stuff for us is that some settings are purposely blocked by our administrator, and this appears to be one of them. At least, I think it is…I didn’t actually see any “App passwords” language, but I’m guessing the equivalent thing is called “Passkey” for us…
@dethompsOWU When I try using my gmail account password then I get a specific error message regarding an application password. Your error message is different to that, so may be a different reason for it.
phpList - Cannot create POP3 connection to imap.gmail.com: Can not authenticate to IMAP server: [ALERT] Application-specific password required: https://support.google.com/accounts/answer/185833 (Failure)
Hrmmm…I’m thinking I’ll try to connect with Thunderbird to the IMAP for the Gmail account and make sure that works…maybe there’s something amiss with the password I have on-file or a “hidden character” (or something) that somehow worked its way into the config…
That way, I can at least rule out IMAP connections to the account in general as being culprits here…
So weird bit of strangeness…I received two “undeliverable” emails (both phpList subscribe attempts) to the new bounces account I mentioned above…after I stopped for the day…
One at 5:46 p.m. and the other at 8:34 p.m. (My EOD was 5 p.m.).
But the phpList back-end is still giving me the “Cannot create POP3 connection to imap.gmail.com…” error that I mentioned earlier.
So it’s like it’s partially working, but not all the way (or something)…
So here’s another bit of weirdness…I just got a bounceback the old way, meaning something worked with the new for a few hours last evening but now isn’t for some reason (despite me not doing anything with it at the time…
So here’s what I’d ultimately like to do once this is all working (I’m spelling this out in case there’s an alternative to the way I’m attempting to do it):
An email bounces back as undeliverable for whatever reason (be it “full message box” or “account no longer exists” or whatever).
phpList automatically blacklists that email address.
Currently, I have to manually do the blacklisting (which is a chore, seeing as how “process undeliverables” is pretty low on my priority list, so I’ve accumulated 171 “undeliverables”.
This is made especially more annoying now that spammers have discovered our newsletter’s subscribe link, but so far they’ve been thwarted completely by phpList’s whole “thanks for registering…just click the link in the email we just sent you to verify your account” process, since they’re providing non-working email addresses.
Currently, we have no CRON jobs running on the phpList server, so I may still need to “Process Bounces” in the back-end periodically or whatever…but surely that would be easier/faster to deal with than having to do each one one-by-one like I do now…
I enabled “2-Step Verification” on the Google account that I’m using for this, and now the Manage Bounces → Process Bounces error is different:
Cannot create POP3 connection to imap.gmail.com: Can not authenticate to IMAP server: [ALERT] Application-specific password required: https://support.google.com/accounts/answer/185833 (Failure)
Identifying consecutive bounces
Nothing to do
total of 0 subscribers processed
Namely, the thing about App Passwords is in there, complete with a link.
The problem being that I had our IS guy look into it, and from what we can tell Google is in the process of removing that functionality…at least for Google Workspace Education group Google Accounts (as opposed to the individual Google Accounts that people can get for free)
Yet another new data point that’s weird…despite the error message mentioned above, I just received a bounce email from phpList in the Gmail account I’m using for this…
So it’s like it’s working, but when I click the “Process Bounces” button in the back-end that doesn’t work. And maybe I’ll never need to do that. My plan is to set up bounce rules that blacklist stuff…can you tell I’m (mostly) in the dark on this stuff?
Yes that is expected once you set the $message_envelope variable in config.php
You need to get the bounce processing working in order for phplist to automatically deal with bounces. I can get that working with my personal gmail account but have no knowledge of how google apps is different. You just need to get someone to explain to you how to retrieve emails from the gmail account.
The other factor worth fixing is installing the php IMAP extension. The lack of that is why you have had to install the IMAP2 plugin, but there is no reason why the extension should not be installed.
Let me look into getting that php-imap extension installed. This will probably be a few days, because we’re quickly approaching the date each month that we use phpList to send out our monthly newsletter. That date is next Tuesday, and I don’t have time to devote to this in any significant way before after then
Ugh so we have a wrinkle…and this is why we didn’t have the php-imap extension in the first place.
Our head IS guy looked into it, and had this to say:
When we tried this back in February, we couldn’t install php-imap on our phpList server because that server is running Enterprise Linux 8, and the php-imap extension isn’t included in any of the standard EL 8 distribution package repositories. As a policy, we don’t install out-of-distro packages or package repositories on OWU servers. Anyway, I just checked, and Rocky Linux 8.8 (which is what’s on our phpList server) still doesn’t offer the php-imap module.
So there you have it. That’s why we don’t have php-imap.
I see that the Gmail account can have POP enabled, though, so maybe we can use that instead? If that is an option, are there any special config.php settings I need to know about to make that work with Gmail?
@dethompsOWU I’ve had success with POP for the Bounce emails. You MUST use either an App Password fo the gmail account, OR use the (Now) OAUTH setup in the second window. POSTSMTP (a Wordpress plugin) has instructions for creating the app password for use in the password field. Again, with 2FA for gmail you have to use an app password or OAUTH2 passcode.
When our IS person initially gave me the App Passwords link (https://myaccount.google.com/apppasswords) an error message came up basically saying that the feature was disabled.
On a lark, I tried it again just a few minutes ago, and surprise surprise it worked this time! The screen that came up had a place that let me set up a new App Password, which spat out a 16-character password, which I put in the phpList config.php file. Manage Bounces → Process Bounces worked after I did that…no erroring out. Now to figure out what it did, as I didn’t actually configure any “bounce rules” or anything yet…
So now that I have “Process Bounces” working, what I want it to do is blacklist all bounces…we’ve found that our bounces are either spam attempts or “mailbox is full/account no longer exists”-type things, neither of which we want to keep around (but we’d like to prevent those same accounts from attempting to subscribe again.
We’re a university, so if there is a legit “mailbox is full” issue, the individual alum will let us know, at which point I can “unblacklist” their email address and all will be fine
I’m not sure what it’s doing when it does “Process Bounces”, but I can say that it appears that the email address appears to not be removed from phpList nor is it blacklisted…
This (Bounce Management | phpList manual) talks about “Advanced Bounce Management” and using RegEx and whatnot to configure it to do different things depending upon what the bounceback is, but in our case, we just want to blacklist all bounces…surely that doesn’t require RegEx and whatnot?
K nevermind. I did some further digging and determined that I just needed to define a Bounce Rule that looks like “(A COUPLE WORDS FROM THE BOUNCE’S ERROR DESCRIPTION|A COUPLE WORDS FROM THE BOUNCE’S ERROR DESCRIPTION|A COUPLE WORDS FROM THE BOUNCE’S ERROR DESCRIPTION)” and set that to blacklist the email address. It took a bit of back-and-forth “Check Bounce Rules” for me to catch all of them, but I eventually did and so all is well now as far as I can tell
