Hey guys,
we encounter a strange problem with phpList 3.3.1: It doesn’t send all emails, but only parts of the recipients. It’s always a different amount of emails, that isn’t sent, in total numbers and percent of the entire list.
One list was 10,692, and 2,291 were not sent, and in another list there were 4,499, and 128 were not sent. The messages remain in the queue of unsent messages, and if we try to send the remaining, it seems to work, but actually does nothing.
It’s not a problem with the blacklisted recipients, as they rare already taken out.
I would be happy for any advice!
Thanks!
The only time I have had these kind of issues is when an email address had an unusual, unallowed character in it (in my case, these characters were not displayed, and ‘unprintable’). The system would hang on that email address, and would stop sending.
So you might want to export the list of unsent email addresses, and using a program like ‘vi’, edit the file, and look for unexpected characters.
Once I deleted the offending email address(es), everything worked as expected.
I already found an answer in another thread. I had a wrong configuration in my config.
But I will check this as well. I thought, email addresses with wrong characters were ignored?
Thanks!
Unfortunately the issue is still there.
The system was sending out the first 10.000 emails as quickly as usual (3.500 mail / hour), and now hangs with 1 email every 45seconds. Plus it already has more than 300 emails that could not be sent (why??? Can I see, why?) and will be sent again later (which never happens, but fills the database with unsuccessful send attempts)
One more thing about this: When I’m on the messages page on the active messages, it says, that’s sending out with a speed of more than 12.500 mails per hour (which is impossible), but the number of messages to be processed is cycling between 3.960 and 3.600, then going back to 3.960…
And another update:
The log shows “Could not instantiate mail function”, when running on php mailer, or “Requested action not taken: mailbox unavailable invalid DNS MX or A/AAAA resource record”. Why is this blocking the process, and not being handled by bounce management?
It sounds like your mail server is restricting connections, and that you need to reconfigure or change your mail host. Limitations on messages and connections per hour are typical.
The sending speed of 12,500 refers to the speed at which phplist can generate and process messages after which they are passed to whichever sending backend you have selected. It does not reflect the speed at which messages are leaving your server necessarily. Just because 12,500 messages can be processed per hour does not mean they can actually be sent per hour.
Hi Sam,
Thanks for your email!
The server has limitations, and the settings are 235 email per 240 seconds (3,525 emails per hour, the provider’s limit is 5,000). That’s why I was wondering about the strange number of 12,500.
It is more a problem of DNS, because now I sent out another smaller list, of which 56 emails bounced with a DNS message. And although I set up a bounce rule for this, it is constantly being ignored. This is my thread about this: https://discuss.phplist.org/t/mailbox-unavailable-invalid-dns-mx-or-a-aaaa-resource-record/3890
However, do you know, how PHPList sends out email? Let’s say, I have a stack 10,000 email addresses. 9,000 work as they should.
Then I have 300 wrong email addresses. Will PHPList get stuck with this stack (as it is bigger than the batch size) and will constantly try to send the wrong emails, or does it skip this batch and move to the next (to return later)? If first, this would explain the speed figures.
Best,
Simeon
@sostberg Sounds like you’re edging closer to the core of the problem.
phpList supports a variety of sending backends, such as Amazon SES, local or remote SMTP. The backend you’re using will determine how undeliverable messages and invalid addresses are handled. I am most familiar with using remote SMTP for sending, and in that case the mail server (e.g. postfix) handles 100% of mail delivery/failure once the messages are received from phpList. phpList statistics are independently collected from the mail server handling in such cases, except for bounces.
Others here are better placed to advise regarding the behavior of other sending backends; I’m not sure if bottlenecks can arise when using built-in PHP mail functionality instead of an external SMTP backend.
Hi Sam,
Thanks for your email!
I’m also sending with remote SMTP, not via phpmailer. My host is 1&1.
Do you have any hints for me, why the bounce rule is ignored?
Thanks!
@sostberg As I recall, when phpList cannot make the connection to the SMTP server then it will keep trying to do so and will not consider messages ‘processed’ until it succeeds. Therefore the processing numbers either continue to increment upwards (you can end up with many more messages ‘processed’ than subscribers exist), or the ‘processed’ counter resets to zero and continues in a loop. I’m not sure of other reasons why the ‘processing’ numbers would be wrong. I recommend you check whether your SMTP provider is reaching a limit and rejecting further mail / connections, which could explain the problem.
@samtuke
I have this processing numbers issue: I only have around 3,000 emails in the list, but more than 20,000 were processed.
So, I’m wondering, how to avoid this. Is it a way to change the batch size? Let’s say, I have 300 wrong emails. If my batch is 235 mails in 240 seconds, it get’s stuck with the wrong emails. But if I change the batch size to 3,525 in 3,600 seconds, I have the same amount of emails sent per hour, but the system at least sends out 3,225 emails (given the 300 wrong addresses)?
@sostberg You might be misled thinking that invalid / unreachable addresses are the cause. Your SMTP server should be handling failed deliveries to /remote/ hosts just fine, if it’s properly configured. I suspect the issue is more likely to be connection problems between phpList and your SMTP server. To restate: not connections from your SMTP to the Recipient’s SMTP, but from your phpList server to your SMTP server. That is more likely to be where the message processing is not being completed according to phpList, and hence why additional (never-ending) ‘processing’ attempts are being made.
There could be other explanations though; I’m not sure. I recommend you check your logs carefully to determine what mail is not being ‘processed’ and why.