Phplist & rss lists: Cannot get these to work. Poor/nonexistent documentation

Hi there,

I’m at my wits end attempting to generate an automatic mailing list generated by an rss feed. There is little to no documentation or useful blog posts on this subject matter, as all of the documentation seems to be for phplist 2.0, whereas I am attempting to use phplist 3.0 with the rss plugin (which hasn’t been updated for a couple of years and has no documentation itself).

Thus far I have been able to:

  • Subscribe users using the subscribe pages
  • Send test messages
  • Send messages to the “test” list
  • Installed the rss plugin
  • snatched/retrieved rss items
  • altered the config.php file to turn on repeat scheduling (define(‘USE_REPETITION’, 1); ) - though due to the poor documentation, I don’t know what preciously this is meant to do or whether it has had any effect.

I have been unable to:

  • schedule/send any emails that use the RSS feed
  • get the mails to send if I add new items to the RSS feed

Could someone please provide me with a step-by-step procedure of how to construct a mailing list that will:

  • send a new email to the list when a new rss item is found
  • check the rss items hourly for new items (I understand this can be done via a cron job; there is nothing in the documentation that I’ve seen that tells me what command to use to do this however; if there is it’s probably for 2.0 so god knows if it works with 3.0 and the plug in.)

And finally, does anyone know whether it is possible to process the RSS feeds themselves via the template system? (As I’ve not been able to send an email via RSS yet, I have no idea what format phplist will reproduce the RSS items in, so I have no idea how I can/will display the feed items in the email. I’d obviously like to have more control than having to just reproduce the description or title tag, such as being able to place the text of particular RSS tags in particular locations, with specific formatting etc.)

Sorry if this is a bit long and if I’ve actually over looked something. I’m very frustrated with phplist right now and I’m very close to giving up on it.

Thanks for your time and help,

@confusedpublic You should use the Rss Feed plugin

@duncanc, I have installed the plugin, as I have said in my post. Thank you for linking me to better documentation on the plug in. I’ll report back if this helps.

@confusedpublic I’m a bit puzzled as how you managed to find and install the plugin without using the documentation. Please can you explain?

I didn’t/couldn’t find that particular page or some how managed to overlook the bottom half of it. I found the github, and an older version of the plug in page, probably linked to by blog posts.

When searching for how to generate the emails from rss lists I was pretty much only returning results for phplist 2.0, and the 2.0 documentation.

@duncanc, I’m working through the example, and immediately there are mistakes in the documentation.

  1. When one creates a list, the list must be given the RSS feed link. There is no where on the RSS tab when creating a campaign to enter an address as the documentation claims.
  2. There is nothing in the documentation on the actual options given in the RSS tab, namely the “frequency it should be used”, which I take to mean the frequency with which the RSS feed should be checked? But this just seems to be a doubling up of the scheduling option to repeat the list. I chose “hourly” here to match the repeat option.

Another problem I am having is that the plug in is inconsistent at best when it comes to fetching new items. Deleting the rss file and recreating it does not lead to new items being grabbed, and with the old items being less than a day old they are impossible to delete.

Good news, however, I seem to have successfully sent some emails now. I have no idea why it has worked now but didn’t before… Possibly not aligning all of the RSS timings (some might have been set hourly, some daily).

Anyway, thanks for directing me to the correct(ish) documentation!

@duncanc just realised you appear to be the author of the plug in. In which case, thanks!

It seems to be working reliably and frequently now, so time for another question: how do I get namespaces to work properly? Placing a colon into the replacement namesakes, e.g [namespace:tag], results in “[namespace:” being printed in the message. I had a quick look through the php files, and as far as I can tell the functions replaceProperties and generateItemHtml are responsible for swapping the tags for the rss items, but I can’t see anything in these functions that would indicate a colon would cause a problem.


@confusedpublic There are actually two RSS plugins

I pointed you at the one I developed, the RSS Feed plugin, as it sounded like you had installed rssmanager, but you were adamant that was the one you had installed.

Your new comments confirm that you are using the rssmanager plugin. I suggest removing that and installing the RSS Feed plugin.

Well that was completely unclear. I’ll switch over now. Thanks for your patients. Frustration has gotten the better of me with getting this to work.

@duncanc I think we’re nearly at the end here. Plug in seems to work nicely, repeats properly, grabs the right items, etc. now.

I’m still struggling with getting custom tags with namespaces to display any content. I have a custom tag in my feed, with the namespace cppopvault, the tag is img. It’s a url for an image related to the rss item. I’ve had a look through your two plugins code, and as far as I can tell your Plugin-Common parses the namespace tags, but the RSS Feed plug in does not do anything with this.

I can’t, however, figure out how the plug in works, at least not enough to alter it to add the custom tags as replaceable strings (e.g. [cppopvault:img]). Allow me to ask some questions/describe how I think this plug in works, and perhaps we can work through his together.

The getTag() function of CommonPlugin/vendor/fguillot/picofeed/lib/PicoFeed/Parser/Item.php grabs a particular tag, and the comments say it can grab namespace tags. This would be the function to use. It requires a tag to be specified however (the $tag variable to be declared). This is fine if we can get it an array of the custom tags, either manually entered on an options page, or automatically through the script itself.

The addItemData() function seems to add the relevant info for each item to the database, and as such isn’t really set up to allow for any tags other than the standard ones listed here. Adding custom tags to this plug-in would require a rewrite for them to be added to the database (and therefore require more work than simply adding an option or line to an existing function.

My approach to adding custom fields, given the above, would be to have the user specify them on the RSS tab with two inputs. One for the namespace, the other for the tag. This would allow the fields to be added easily to the database, allow the parser to check that this name space is correct, and then select this info from the RSS feed if these tags exist.

Am I on the right lines here? Or have I missed something obvious, and I should just be using a different way to place a namespaced tag into the HTML template?

@confusedpublic Rather than add fields to the RSS tab I think that adding a settings page entry will be simpler. Specify one custom tag per line


etc. These then would be global to all RSS feeds but that probably isn’t going to be a problem.

But cppopvault is a namespace prefix not a namespace. picofeed seems to deal with prefixes but calls them namespaces. So long as you always associate the same prefix with the same namespace then it should work.

Then the get processing needs to look for these elements in each item and include the values in the array used in the call to addItemData() with a key of cppopvault:img and value of the elements content.
You should then be able to include a placeholder such as [cppopvalult:img] in the RSS and it should get replaced automatically.

1 Like

@confusedpublic Can I have the URL of your feed? I have made some changes to handle namespaced tags but am having trouble finding any feed that uses those.


Also, for further testing, any of the feeds include 2 namespaces. (They’re mostly not updated any more I think, but provide a decent test case I guess?)

(Yeah, sorry about mixing up the terminology. I struggled to grasp namespaces when parsing podcast xml feeds, so my knowledge is at the level of “I’ve made it work but I’m not sure how or why”!)

@confusedpublic Thanks. I note that the feed generator is using an invalid format for the publication date. It should be something like

<pubDate>Sun, 19 May 2002 15:21:36 GMT</pubDate> 


<pubDate>21:11 17-08-2016</pubDate>

Picofeed is rejecting that and using the current date/time instead.

Also the author tag is in the wrong place. It can be within an item not within the channel element.

I’ve now adjusted the XML.

I probably won’t have a chance to test the changes you’ve made until Monday, hope that’s okay? Cheers for this.

@confusedpublic I have made some changes to support custom tags that you can install from a different branch
Just install using that url on the Manage Plugins page. You also need to upgrade Common Plugin first.

On the Settings page the RSS group now has field where you can enter custom tags, one per line. They need to be exactly as they will appear in the RSS feed, so cppopvault:img in your case.

You can then use the placeholder [cppopvault:img] in the RSS template.
As it is the url for an image then you will want to add an img element whose src attribute is the placeholder.

I have also added a field to the RSS tab for a custom template that applies to that message only.
If you handle multiple RSS feeds then you won’t want the general RSS template to have the placeholder for feeds in which it doesn’t appear.

1 Like

Doesn’t look like it’s worked. There are rows in the RssFeedPlugin_item_data table for the ‘cppopvault:img’ tag, and these are populated properly, yet the email has

<img src=3D"[cppopvault:img]" style=3D"width: 296px; height: 212px;" />"

in its source still. (Mac Mail shows those "3D"s between = and " in the html source of emails for some reason) So PicoFeed is scraping the feed properly and populating the db correctly, but something is going awry when the plugin attempts to fill in the template.

I can’t find a log file anywhere and don’t remember seeing any logging functions in the codebase. Is there anything that reports on success/failure of replacing the placeholders for RSS items?

(I’m hoping it’s not failed due to a typo, and I’m checking several times)

EDIT: Wait, looks like the rss was generated improperly.

@confusedpublic This is my custom template

generates this in the email

The error was in the RSS file; I’d forgotten to include to the namespace tag when the new items were amended to an existing RSS file.

The replacement now works!

(Only problem is that I can’t seem to actually see the images. They just don’t seem to be getting displayed. Which is confusing as the links are correct. Copying them and putting them into a browser is correct. My custom template is:

<div style="width:600px; height: 212px;">
  <div style="width:296px; height: 212px; position: absolute; float: left; left: 0px>
    <img src="[cppopvault:img]" style="width: 296px; height: 212px;"/>
  <div style="width:304px; height: 212px; position: absolute; float: left; left: 296px; padding-left: 5px; margin-left: 5px;">
  <h3><a href="[url]" >[title]</a></h3>

I’ll try again without the CSS/style attribute and report back.

Thanks for all your help in this, looks to be working great now.

Taking out the style attributes did allow the image to show up. I’ll try a few things tomorrow and let you know how it goes.