I recently updated to v3.6.12 using Softaculous on our cpanel and have since got 403 access forbidden alerts but ONLY when navigating through the “subscriber lists” using the navigation buttons (see attached screenshot with red arrow pointing to these buttons). Could this be caused by the update and does any one have any idea how I can resolve this?
Also on the same screenshot, some of the lists have a spinning phplist logo that says “please wait, processing your request” when you hover over them - What’s going on there and is this something to do with the 403 access forbidden alerts?
And finally, in the process of trying to resolve this I have upgraded the php on the server to php 8.0 - can anyone confirm whether phplist v3.6.12 works with php 8.0
@phillegg The 403 code is probably caused by anti-malware on the web server, such as modsecurity. You will need to look in the web server logs to try to find the cause.
There is a change in phplist 3.6.12 to calculate the subscriber totals for each list asynchronously, which causes the spinning logo.
Hi Duncan. Went back to the hosting company with your suggestion and they came back saying this:
Originally we were seeing Modsecurity rules flag up, thats what the firewall rules we mentioned on our call are. However, since whitelisting the rules the issue has continued! The rule we whitelisted looked like this.
From what we can tell on our side, the errors aren’t being triggered from Modsecurity, or from a permissions issue. We’re unable to see why the 403’s are actually occurring at the moment. Are you able to advise Duncan that Modsecurity has been ruled out and see if he has more suggestions?
Any ideas on what we can look into now, since the hosting compnay has whitelisted the Modsecurity?
@phillegg Your main web site is Wordpress. Are there any Wordpress plugins that might be causing this? My site has Wordfence which sometimes causes false errors with phplist.
@phillegg In the first screenshot you posted, there is the logo “Zen”. What is that? Presumably that has generated the content of the page returned to the browser.
Also, to clarify, the problem doesn’t happen on every click of the navigation buttons, but only intermittently. Is that correct? And does the same URL, with the same value for the start parameter, either always fails or always works?
Can you use php 8.1 or 8.2? The 8.0 version is no longer current so there is no real reason to use it in preference to a later version.
A: You are correct: it doesn’t happen on every click of the navigation buttons, but it does seem to only happen on the ‘Subscriber Lists’ page navigation buttons - generally within 2 or 3 clicks & occasionally on the 1st click (of the navigation buttons)
B: The start parameter seems to be different each time it (intermittently) fails.
The MultiPHP Manager on the cpanel only offers up to 8.0 - have asked hosting company about offering 8.1 & 8.2
Having spoken to the hosting company (Zen)'s support team they came back & said:
The only thing we can see trigger from the office IP now appears to be a “client denied by server configuration” error. This sort of error pops up when a .htaccess is denying access to an IP address due to a rule set in place on the htaccess file
I’d probably recommend if you can checking your htaccess files in the file manager/ftp and seeing if the forum has an opinion on how the rules set in place might affect your phplists site.
As for Php 8.1 and 8.2 we currently don’t yet have these enabled, the highest we offer is 8.0, this will likely change in the future to offer newer Php versions but at the moment you can only use 8.0`*
With that in mind I’ve pasted in the code inside the .htaccess file for the top level of the phplist folder below. Thing is, this is currently named .old_htaccess. Not sure why and more to the point is this a problem? Also, do you happen to know whether the phplist update I did the other day may have renamed this file automatically in this way?
‘.old_htacess’ file (note the name) content is as follows :
DirectoryIndex index.php
<FilesMatch "\.(php|inc)$">
# Apache < 2.3
<IfModule !mod_authz_core.c>
Order allow,deny
Deny from all
Satisfy All
</IfModule>
# Apache ≥ 2.3
<IfModule mod_authz_core.c>
Require all denied
</IfModule>
</FilesMatch>
# api.php is the entrypoint for phpList4
<FilesMatch "(index.php|dl.php|ut.php|lt.php|download.php|connector.php|api.php)$">
# Apache < 2.3
<IfModule !mod_authz_core.c>
Order allow,deny
Allow from all
Satisfy All
</IfModule>
# Apache ≥ 2.3
<IfModule mod_authz_core.c>
Require all granted
</IfModule>
</FilesMatch>
<IfModule mod_rewrite.c>
Options -MultiViews
RewriteEngine On
RewriteRule api/v2/ api.php [L]
</IfModule>
# if you want more than this for attachments, you can increase these values
# but you really, really should consider uploading them somewhere
# and only sending a link in your mails
# bigger than this really doesn't make sense in emails
## these lines are now commented out, to avoid 500 errors, https://mantis.phplist.com/view.php?id=9937
## but you should consider adding them
#php_value upload_max_filesize 3M
#php_value post_max_size 4M
@phillegg That looks to be the content of the .htaccess in the phplist distribution top level directory, lists2 in your case.
There should still be a .htaccess file, otherwise I would guess a lot more would not work.
Given that the problem is intermittent, I don’t think that .htaccess is the cause, otherwise the problem would be consistently occurring.
This might not be the cause but the web site seems inconsistent with URLs starting with “www.”
https://futureproofpromotions.com/lists2/admin/ and https://www.futureproofpromotions.com/lists2/admin/ both seem to be valid. It is probably better to redirect one of them to the other. That can be done through cPanel I think.
Have redirected the ‘futureproofpromotions.com’ to ‘www.futureproofpromotions.com’ as suggested but still getting the 403 (as was thought might happen). The hosting company suggested I try including an ‘allow from (my ip address)’ in places where it says deny from all in the .htaccess file like so:
<IfModule !mod_authz_core.c>
Order allow,deny
Deny from all
Satisfy All
becomes…
<IfModule !mod_authz_core.c>
Order allow,deny
Deny from all
Allow from xx.xxx.xxx.xx
Satisfy All
Does that sound like something worth trying in your opinion or do you think that may cause more issues? (a bit nervous about trying things out on a .htaccess file)
Otherwise, the hosting guys don’t know what to suggest but say they’re going to have a chat with some other of their support team & come back to me.
Going to have to wait for php 8.1/2 to be available, otherwise would like to hear your opnion on the suggested .htaccess edit above.
@phillegg That section of .htaccess is handling the case of apache version earlier than 2.3. I think that your server should be running version 2.4.
One other possibility is that the http requests made by the new asynchronous code displaying the subscriber totals for each list is interfering with the request made when clicking the navigation button. Does the problem happen if you wait until the subscriber totals have been displayed for each of the, in your case, 10 lists?
Will ask hosting company regarding apache 2.3 to 2.4 & see what they say.
Otherwise your other question regarding the asynchronous code interfering: to be honest I’ve not been able to get a page to settle - i.e. the spinning logos continue to spin forever (usually for 5 or 6 of the 10 lists displayed!) Maybe that’s the issue causing these 403s?
btw: what do you mean by “asychronous”? I googled it & still not 100% sure what it mean but presume it’s something like live updating of totals maybe? tbh the totals always displayed fine for me before? What’s gained by displaying the totals this new way?
@phillegg Before that change the totals of subscribers for each list were calculated as part of producing the page. For lists with large numbers of subscribers, I think in the millions, that apparently caused a delay in displaying the page, which is annoying if all you want to do is navigate to one particular list.
The new method is to display the page without the totals and then send a sequence of requests each for the totals of one list. There is only one request at a time so that might take a few seconds depending on the number of lists, but you can navigate away from the page as soon as it is displayed.
I could take a look at your page to see whether anything looks to be going wrong, if you are happy to give me access. If so, send me a private message by clicking on my image.
@phillegg The problem seems to be caused by the sequence of 10 requests for list subscriber totals. Usually the last three or four fail with the 403 error. Then navigating to the next page may also cause a 403 error. Possibly there are too many requests being sent in too short a period of time, and it looks like an attempt at “denial of service”.
You can reduce the number of lists shown on the page, which will reduce the number of requests, by adding this setting to the config.php file
define('LISTPAGE_MAX', 10);
10 is the default value, so maybe try 5 to see whether that improves things.
If you want to go to a specific list then you can enter part of the name (such as folk) in the search box rather than paging through the lists.
Sorry, I’ve only just seen your latest response (was mistakenly looking on the private string). Anyway many thanks for having a look - much appreciated.
Moving forward: I’ve restricted the lists to 5 a page like you suggested but still the list totals are getting stuck far too often and I get the spinning logos that never resolve (although in terms of getting 403s it is better- in fact I’d say if all the spinning logos resolve then I don’t get a 403 - but that only happens 50% of the time)
Surely the issue here is that this new way to generate list totals is getting stuck a lot of the time and (probably) causing issues. If that was sorted - I think these 403s may disappear entirely.
Can I ask… are you part of the development team (or do you know who is?)
I don’t think it is possible to easily revert to the previous behaviour but you could try adding a delay, similar to that change, to see whether that improves the result
The value 250 is the delay in milliseconds, so that would be 0.25s.
I’m not part of the phplist development team but do submit changes and develop plugins, so am probably the most active outside of the main developer.
If you are able to, it might be helpful to raise this with the hosting company as it is clearer now what is causing the 403 errors - the sequence of requests. That might help them to see what is rejecting the requests.
OK thanks for that. Will investigate adding that delay like you suggested and go back to the hosting company to see what they say now we have a clearer idea as to what’s probably happening here.
While I do that though, are there no other people within the entire community who are having the same issue since this update? (couldn’t see any similar discussions on the forum)
I experienced the 403 error when switching between composition tabs since the last message sent with 3.6.12. The only change I can see is PHP was upgraded from 8.1.16 to 8.1.17. I found a suggestion to turn off ModSecurity for the domain which solves the problem. After editing I enable ModSecurity again.
