[SOLVED] Conflict between MAILQUEUE_BATCH_PERIOD and MAX_PROCESSQUEUE_TIME

ℹ️ Support
1

I’m finding that it is not easy to create a campaign that will send out a limited number of emails per MAILQUEUE_BATCH_PERIOD so that I can slowly send out a campaign over a longer period of time, like a month.
In the following experiment I’m trying to spread out the sending of only 6 emails over 3 hours. I know this can be done simply by setting how many emails to send per batch period, but this won’t work if you want to send only between say 9 to 5 pm and pick up the next day between 9-5. So I tried an experiment:

For example, let’s say we have a cron that fires off every half-hour. Let’s further say our MAILQUEUE_BATCH_SIZE is 2 (this is an experiment) and MAILQUEUE_BATCH_PERIOD is 55 minutes and MAX_PROCESSQUEUE_TIME is 50 minutes. For this experiment only 6 emails are included in the list. According to how MAX_PROCESSQUEUE_TIME works, processqueue should stop at 50 minutes (and it does as I’ve examined the processes on the server and it dies right at the mark), and the next cron job at the half hour mark should start the process over again.

My tests have shown the first cron job firing off the processqueue goes as expected and 2 emails are sent in the first 50 minutes. However, subsequent crons produce a report that seems to indicate that phplist processqueue goes back and calculates how many emails were sent in the last sending period and then deducts that amount from the next-up cron job so that instead of 2 emails going out only 1 does. See the messages below:

At 2:30 the Cron fires off and the emails are running and out of the gate.

At 3:20 pm (50 minutes after cron fires) phplist has done exactly what it is supposed to do: sends out 2 emails in 50 minutes, so at 3:20 sending stops. And the following confirmation message is received:

[Wed 25 Jul 2018 14:30] [CL] Started
[Wed 25 Jul 2018 14:30] [CL] Sending in batches of 2 emails
[Wed 25 Jul 2018 14:30] [CL] Processing has started,
[Wed 25 Jul 2018 14:30] [CL] One campaign to process.
[Wed 25 Jul 2018 14:30] [CL] Processing campaign 3
[Wed 25 Jul 2018 14:30] [CL] Looking for subscribers
[Wed 25 Jul 2018 14:30] [CL] Found them: 6 to process
[Wed 25 Jul 2018 15:20] [CL] batch limit reached: 2 (2)
[Wed 25 Jul 2018 15:20] [CL] Script stage: 5
[Wed 25 Jul 2018 15:20] [CL] 2 messages sent in 3000.02 seconds (2 msgs/hr)

Now, at 3:30 pm the cron job runs again and 2 emails are supposed to go out this period, but only 1 does with the following message:

[Wed 25 Jul 2018 15:30] [CL] Started
[Wed 25 Jul 2018 15:30] [CL] Sending in batches of 2 messages
[Wed 25 Jul 2018 15:30] [CL] This batch will be 1 emails, because in the last 3,000 seconds 1 emails were sent <------- problem here.
[Wed 25 Jul 2018 15:30] [CL] Processing has started,
[Wed 25 Jul 2018 15:30] [CL] One campaign to process.
[Wed 25 Jul 2018 15:30] [CL] Processing campaign 3
[Wed 25 Jul 2018 15:30] [CL] Looking for subscribers
[Wed 25 Jul 2018 15:30] [CL] Found them: 4 to process
[Wed 25 Jul 2018 15:55] [CL] batch limit reached: 1 (1)
[Wed 25 Jul 2018 15:55] [CL] Script stage: 5
[Wed 25 Jul 2018 15:55] [CL] 1 messages sent in 1500.00 seconds (2 msgs/hr)

It would seem to me that processqueue should not look back and calcualte how many were previously sent if the purpose MAX_PROCESSQUEUE_TIME is to kill the proecessqueue and start from scratch. See the definition of MAX_PROCESSQUEUE_TIME, below.

// MAX_PROCESSQUEUE_TIME
// to limit the time, regardless of batch processing or other throttling of a single run of “processqueue”
// you can set the MAX_PROCESSQUEUE_TIME in seconds
// if a single queue run exceeds this amount, it will stop, just to pick up from where it left off next time
// this allows multiple installations each to run the queue, but slow installations (eg with large emails)
// set to 0 to disable this feature.
define(‘MAX_PROCESSQUEUE_TIME’, 0);

Or maybe I’ve mis-configured something?

2

Hi @pancakehollow ,
i’ve your same problem. I’ve set mailqueue_batch_size 200 and mailqueue_batch_period 900 but this is the result:

This batch will be 18 emails, because in the last 900 seconds 182 emails were sent

Cronjob running every 900 secs (i’ve also try to set mailqueue_batch_period to 899 but the result is the same).

I’ve also set MAX_PROCESSQUEUE_TIME to 0 but the problem is already running.

Have you solved? And how?

Thank you.

3

@vassiskansa When you use a cron job then the way that phplist uses MAILQUEUE_BATCH_PERIOD is not really useful because it can reduce the number of emails to be sent in each batch to be less than the batch size.

To avoid this you need to set MAILQUEUE_BATCH_PERIOD to a small value, say 5 seconds. This should “fool” phplist into sending the full batch size.

4

Hello. The way I solved this was to use the scheduling tab and the re-queue feature. It works great for my purpose.

I had a drip campaign that needed to only send out like 40 emails per day over a period of a month. I created a separate install directory for these campaigns although I don’t think that is required. So I used the following settings in my config.php and set the cron job to run everyday just after the time I input into the scheduling tab to run the queue. So if the scheduling tab kicked off every day at 8 a.m. I ran my cron job daily at say 8:05 a.m.

Then I used the following settings in my config.php:

define(‘MAILQUEUE_BATCH_SIZE’, 40); <---- this worked and spread the 40 emails out over 8 hours.
define(‘MAILQUEUE_BATCH_PERIOD’, 28800); <— this equals 8 hours
define(‘MAILQUEUE_AUTOTHROTTLE’, 1); <---- this seemed to be required.
define(‘USE_DOMAIN_THROTTLE’, 0); <---- this disables to next two lines. I seemed to need this, too.
define(‘DOMAIN_BATCH_SIZE’, 1);
define(‘DOMAIN_BATCH_PERIOD’, 120);

Once I set this up I didn’t go in and mess around with any other settings or create any other campaigns in this install of phplist. I might be able to but I didn’t want to take the chance.

1 Like