Processing of queue is very slow

I am running phplist-3.0.12.
My browser is Chrome 46.
My install is at http://www.allinallgone.com/lists/

This is my first experience using phplist.

My list consists of 1100 email address. I am using Mandrill’s SMTP servers. All test emails worked without a problem. I launched the campaign via my browser … not the command line.

The queue has been processing for about 1 hour now. Exactly 356 emails have been processed. The queue seems to process in spurts of 50 and then stop for a period of time. On average it sends 1 email every 10 seconds.

This seems excessively slow to me. Is this rate of processing by design? Is there any way to speed up the processing to 1 email per second?

It’s sending at that speed due to one of these:

  1. Your phpList config.php file is sending speed is set to send at that rate
  2. Your hosting service only allows you to send at that rate
    or
  3. Mandrill’s SMTP servers will only let you send at that rate.

How did you set up the sending, through the web interface, or a cron job?

1 Like

Thanks for contributing your time and expertise to this forum Dan.

I sent the campaign through the browser. The campaign finally finished. It took 7 hours 28 minutes 55 seconds to send 1100 emails. I found that the processing would stall … no activity for more than 2 hours. The only way I could keep the process moving was to constantly reload and select the “process queue” button in my browser.

I have configured a VPS server, with cPanel/WHM, specifically for the phplist application. I have reviewed my php.config settings in Tweak Settings > Mail. A setting that would throttle email, Max hourly emails per domain, is set to unlimited.
So, my host is not throttling my output.

I have been using Mandrill for years. Mandrill assigns a reputation rating based on past performance and I have an excellent rating. I am certain they are not throttling me because I use them with other email applications.

Thanks again

Hi, so that leaves us with option 1.

what is the rate set there?

yes, this is why people use the command line, browsers are not really designed for this kind of thing. However, with 1000 emails you should be fine, once we can work out the issue.

Thank you Anna for your help.

Here is my phplist config.php without comments. Do I need to add a parameter to change the behavior?

$database_host = “localhost”;
$database_name = “REMOVED”;
$database_user = “REMOVED”;
$database_password = ‘REMOVED’;
define(“PHPMAILERHOST”,‘smtp.mandrillapp.com’);
$phpmailer_smtpuser = ‘REMOVED’;
$phpmailer_smtppassword = ‘REMOVED’;
define(‘PHPMAILERPORT’,587);
define(‘SMTP_TIMEOUT’,5);
define (“TEST”,0);
$bounce_protocol = ‘pop’;
define (“MANUALLY_PROCESS_BOUNCES”,1);
$bounce_mailbox_host = ‘localhost’;
$bounce_mailbox_user = ‘popuser’;
$bounce_mailbox_password = ‘password’;
$bounce_mailbox_port = “110/pop3/notls”;
$bounce_mailbox = ‘/var/mail/listbounces’;
$bounce_mailbox_purge = 1;
$bounce_mailbox_purge_unprocessed = 1;
$bounce_unsubscribe_threshold = 5;

If I were to add the following settings, should it correct the problem?

define(‘MAX_PROCESS_MESSAGE’,999);
define(“MAILQUEUE_BATCH_SIZE”,0);
define(“MAILQUEUE_BATCH_PERIOD”,3600);
define(‘MAILQUEUE_THROTTLE’,0);
define(‘MAILQUEUE_AUTOTHROTTLE’,0);

@danwaterloo or @Dragonrider will know best :smile:

The last block of settings would send the message as fast as possible. I’m a big fan of sending messages a little slower…

The size and the period say “send as many as you can in 1 hour”.
I’d probably try setting this to send 1,200 emails per hour… which you can do by telling the system to send 300 every 15 minutes:

define(“MAILQUEUE_BATCH_SIZE”,300);
define(“MAILQUEUE_BATCH_PERIOD”,900);

instead of sending as fast as possible for 300 the three hundred emails, and then waiting for the next 15 minutes to begin, you can tell the system to spread out the sending over the period by putting a delay between one email and the next:

define(‘MAILQUEUE_THROTTLE’,2.5);

puts a 2.5 second delay between one email and the next.

You can look at an online spreadsheet I put together to play around with the settings:
http://www.phplistsupport.com/config-php-setting-batch-qty-period-and-throttle/

1 Like

Thanks Dan.

I added this to config.php:

define (“MANUALLY_PROCESS_QUEUE”,1);
define(‘MAX_PROCESS_MESSAGE’,999);

define(“MAILQUEUE_BATCH_SIZE”,2000);
define(“MAILQUEUE_BATCH_PERIOD”,1800);
define(‘MAILQUEUE_THROTTLE’,0.7);
define(‘MAILQUEUE_AUTOTHROTTLE’,0);

define(‘USE_DOMAIN_THROTTLE’,0);
define(‘DOMAIN_BATCH_SIZE’,1);
define(‘DOMAIN_BATCH_PERIOD’,120);
define(‘DOMAIN_AUTO_THROTTLE’,0);

define(‘MAX_PROCESSQUEUE_TIME’,0);

I setup a test list of 500 email addresses that go to my inbox.I launched the campaign and received 18 emails before it stalled completely.

I switched to using php mail and have a similar problem. It stalled after 49 messages.

Where is your server hosted?
Since you are being slowed down using either an outside smtp server, or the local smtp server (php mail), I’d next look at what the hosting people limit your outgoing mail to… they might limit the traffic on port 25… (outgoing mail)

You should really use either a cron job or the remote queue processing, not try to send using the browser. As you are finding, that often does not work well and the sending rate can be quite unpredictable.
When sending through the browser, the elapsed time for sending is limited to one minute. phplist then stops sending, delays for some time, then should process the queue again. But this doesn’t always work in the way intended. The reliable method is to use a cron job.

1 Like

mm, even for 1000 subscribers though? I am not at all an expert as you know, but I though that would have been okay. I suppose if the batches are small though…

Thanks all. Setting up the cron resolved the issue.

1 Like