I’ve recently setup a mailserver to send out a newsletter to about 20k recepients. This is my setup overview:
configured PHPList to pass all mail via smtp to the local postfix server
no threshold configured, so once the queue is processed, it pretty much should just pass everything to the postfix server
cronjob running processqueue with php-cli every minute, will abort if last processqueue process is still running
Now I’ve already sent out 2 newsletters and both times the campaign was ‘successfully’ processed, leaving about 10k users with status ‘not sent’. My current fix is to just delete the entries in the ‘phplist_usermessage’ DB table which have status ‘not sent’ and then requeue the campaign. it then takes two to three runs with again some users being marked as ‘not sent’ until it really succeeds sending out all messages.
I’ve read here, that only plugins mark messages as not sent. The only plugins I have installed is the fckeditor and the phplist rest api which should really not interfere with sending messages.
I have created some custom bounce processing rules, but, as far as I know, they check only messages I receive in my RETURN email inbox.
I’ve double checked everything, enabled logging wherever I could and have not found the reason for this happening. I’ve uploaded a log of the processqueue cron here.
Does anyone here have an idea how to fix this?
@duncanc
The ‘Lock failed - exit’ entry is the cronjob exiting because there is already an instance of the processqueue cronjob running.
Are there any kind of internal plugins which are not listed under the plugins tab?
@rokkyt phplist itself checks for another instance already running, so there is no need for your cron job to do the same.
Can you enable verbose in your config.php file, then when you process a campaign there will be more, a lot more, lines written to the event log. That might help show what is happening.
@duncanc I’ve sent out another newsletter with the verbose option on. You can download the logfile here. Again it fails to send to about half of the users. Even though there is more information, I cannot see any kind of error, it just stops at one point. There seems to be something that stops the sending out of the mails at about 10k users.
@rokkyt I suggested that you remove the cron job test of whether phplist is already running. It might be harmless but it is one additional thing to have to consider.
Unless there are any errors being reported elsewhere, such as the event log, then I don’t really have anything to suggest.
I found the solution to this problem. I turns out there was another cronjob I was unaware of accessing the database every 15 minutes. During the time that process was quering the DB, the messages which were being sent out at that time were marked as ‘not sent’.