I have searched the forum a little and I could not find any definitive information about how phplist performs and if it can scale up. Has anyone ever produced a benchmark for this? If i was involved in the development i would definitely like to see how each release performs compared to the previous one in a professional CI approach.
What I would be interested in is running a set of big mock list campaigns for small, medium and big mailout content and see in lets say in 1 hour how many emails php list can send to an SMTP server (a dummy one to avoid generating actual email traffic) and also how it performs in terms of memory, CPU, file, db access bottlenecks.
If anything as such exists please let me know.
p.s. I know that the ultimate speed also depends on maybe other factors but as phplist is its own monolith I am really curious to see how it performs.
I am not aware of a benchmark (maybe there is one, but it’s not available publicly?)
I’d be happy to participate in the benchmark, i’ve got my own private cloud of servers that I have built.
In my experience, many things affect the sending speed… my recommendation is to try with a clean database (not too big), and a super fast connection to an smtp server… would be happy to participate.
This does not strictly need a controlled server environment and I think it is great if phplist one day wants to use a similar approach if they have a CI pipeline in place. I can see php 4 is on the way with a composer friendly architecture where base, rest and front end are split.
I will add more parts as per the TODO in the README. Feel free to give feedback
hi @bizmate, i looked at the ‘first post’ on github, and here are thoughts that I have…
I think of phpList as a system of services, all which need to be optimized independently
From my experience, the connection to the smtp server and the database seem to be the major throttling points when sending
There are other issues, such as RAM, disk io, cpu clock speed, etc. that seem to affect the overall throughput.
Might it be better to look at these items ‘one at a time’, in a controlled manner so that we can categorize them in turns of their effect, what can be done to neutralize the unwanted effects?
This is an excellent initiative, thanks, and please keep it going. @nychris is a master of optimisation so he may have points to contribute.
If your objective is to measure changs in performance between releases, Then I think you should pick defaults Which reflect average VPS servers with relatively Meagre resources.
If your objective is to show the maximum possible performance and scalable potential of phpList, then you should use a ramdisk for the MySQL database as that is one of the biggest bottlenecks. Also your configuration of local DNS will be important for lightning-fast lookups.
If you are sending to real remote mail hosts, then you will presumably need multiple IPS to avoid Throttles on those remote service becoming your bottleneck.You can use postfixmulti for that.
My rationale is about seeing how phplist can scale up and indeed i think it could be thought of replicating average VPS specs or also bigger environments.
You can run it locally by following the README and you will notice that
smtp, mailcatcher is a local fake smtp server that just collects emails but does not send them out. My objective is not to test a mailer service too so this isolates this aspect including DNS
phplist takes about 200+ seconds to start, at http://localhost:8080/lists , mainly because i think there is some sort of supervisor or similar setup together with the web service. Too many things together in a single container. I raised some suggestions (might definitely be wrong btw, they are just quick suggestions) in the php-docker project on what it could be improved. No service should really take 200 seconds to set up.
I think i will leave the project there for the next week at least but i might then introduce the next steps i have added in the README .
I mainly need to
set sample campaigns (maybe a small, a medium and a big sized email for each campain)
add X amount users (a lot ideally maybe a million)
process the queue for the whole million users and see how long it takes.
Indeed the DB will be a bottleneck but assuming I, as a user, have set up phplist on an infrastructure that scales like SES or one of the big smtp relay service then it would be nice to know how fast can phplist go considering a limitless run of php + mysql all together. I would not like to use a mysql on a ramdisk as it is something totally unlikely to exist in small/medium setups but can definitely be looked into maybe in the future.
When you say “200+ seconds to start”, are you talking about loading the interface from the time you enter the login credentials
OR
are you talking about the time it takes to send the first email from the moment you start the send?