Awasu » Scraping web pages to RSS
Thursday 1st April 2004 5:54 PM [General]

OK, so it's been a while coming but it's finally here: a plugin that scrapes web pages and generates an RSS feed. While there are web-based services that do this for you (e.g. myrss.com), they are computer-generated and so the quality of the results can vary somewhat.

Well, Allan Wilson has written a plugin that downloads any web page and using regular expressions, extracts the information of interest and generates an RSS feed that you can then plug into Awasu. Full documentation is in the release zip plus a utility that lets you experiment with the RE's and preview the results.

Many thanks to Allan again, he's just a plugin-writing machine! And if you're using his plugins, please think about maybe getting him something from here or putting something in here. Or if you've written a plugin that you think might be useful to others, send it in and we'll put it up on the website.