Use PHP threads. Do the job separately.. in parts... in other words... you can't read all them at once. You can read a little more about php multithreading here: http://blog.motane.lu/2009/01/02/multithreading-in-php/ You can use a non-relational database like mongo or couchdb to manage where you stop and where you have to look back to the RSS feed as well. []'s Regards, Igor Escobar *Software Engineer * + http://blog.igorescobar.com + http://www.igorescobar.com + @igorescobar <http://www.twitter.com/igorescobar> On Sat, Sep 10, 2011 at 10:37 PM, Stuart Dallas <stuart@xxxxxxxx> wrote: > On 10 Sep 2011, at 09:35, muad shibani wrote: > > > I want to design an application that reads news from RSS sources. > > I have about 1000 RSS feed to collect from. > > > > I also will use Cron jobs every 15 minutes to collect the data. > > the question is: Is there a clever way to collect all those feed items > > without exhausting the server > > any Ideas > > I designed a job queuing system a while back when I had a similar problem. > You can read about it here: http://stut.net/2009/05/29/php-job-queue/. Set > that type of system up and add a job for each feed, set to run every 15 > minutes. You can then watch the server and tune the number of concurrent job > processors so you get the optimum balance between load and speed. > > -Stuart > > -- > Stuart Dallas > 3ft9 Ltd > http://3ft9.com/ > -- > PHP General Mailing List (http://www.php.net/) > To unsubscribe, visit: http://www.php.net/unsub.php > >
