oliver.g wrote:I need to check 4,000+ RSS feeds (once daily, and 1,000+ feeds multiple times a day)
We know of at least several sites that are running significantly more channels than this, so you won't have a problem there.
oliver.g wrote:and I need to extract all stories into a single XML file for further data handling. We are talking 10,000+ stories a day.
may be what you want here. These let you extract stories from your Awasu and export them to a file and since it's template-driven, the output file can be in any format you want.
However, XML might start to get a bit unwieldy with that much stuff in a single file so you might also want to take a look at the MySQL channel hook
that stores incoming content in a MySQL database where you can do further processing on it there. Or, of course, you could write your own custom hook to handle the content in any way you choose.