Other good point is: always set a timeout connection when you're getting the RSS data to avoid your thread get stuck unnecessary. Use cURL (is much more faster then file_get_contents). Multithreading in PHP with cURL http://devzone.zend.com/article/3341 Regards, Igor Escobar *Software Engineer * + http://blog.igorescobar.com + http://www.igorescobar.com + @igorescobar <http://www.twitter.com/igorescobar> On Mon, Sep 12, 2011 at 10:05 AM, Igor Escobar <titiolinkin@xxxxxxxxx>wrote: > Use PHP threads. Do the job separately.. in parts... in other words... you > can't read all them at once. > > You can read a little more about php multithreading here: > http://blog.motane.lu/2009/01/02/multithreading-in-php/ > > You can use a non-relational database like mongo or couchdb to manage where > you stop and where you have to look back to the RSS feed as well. > > []'s > > Regards, > Igor Escobar > *Software Engineer > * > + http://blog.igorescobar.com > + http://www.igorescobar.com > + @igorescobar <http://www.twitter.com/igorescobar> > > > > > > > On Sat, Sep 10, 2011 at 10:37 PM, Stuart Dallas <stuart@xxxxxxxx> wrote: > >> On 10 Sep 2011, at 09:35, muad shibani wrote: >> >> > I want to design an application that reads news from RSS sources. >> > I have about 1000 RSS feed to collect from. >> > >> > I also will use Cron jobs every 15 minutes to collect the data. >> > the question is: Is there a clever way to collect all those feed items >> > without exhausting the server >> > any Ideas >> >> I designed a job queuing system a while back when I had a similar problem. >> You can read about it here: http://stut.net/2009/05/29/php-job-queue/. >> Set that type of system up and add a job for each feed, set to run every 15 >> minutes. You can then watch the server and tune the number of concurrent job >> processors so you get the optimum balance between load and speed. >> >> -Stuart >> >> -- >> Stuart Dallas >> 3ft9 Ltd >> http://3ft9.com/ >> -- >> PHP General Mailing List (http://www.php.net/) >> To unsubscribe, visit: http://www.php.net/unsub.php >> >> >
