On Mon, Sep 12, 2011 at 9:37 AM, Igor Escobar <titiolinkin@xxxxxxxxx> wrote: > Other good point is: always set a timeout connection when you're getting the > RSS data to avoid your thread get stuck unnecessary. Use cURL (is much more > faster then file_get_contents). > > Multithreading in PHP with cURL http://devzone.zend.com/article/3341 > > > Regards, > Igor Escobar > *Software Engineer > * > + http://blog.igorescobar.com > + http://www.igorescobar.com > + @igorescobar <http://www.twitter.com/igorescobar> > > > > > > On Mon, Sep 12, 2011 at 10:05 AM, Igor Escobar <titiolinkin@xxxxxxxxx>wrote: > >> Use PHP threads. Do the job separately.. in parts... in other words... you >> can't read all them at once. >> >> You can read a little more about php multithreading here: >> http://blog.motane.lu/2009/01/02/multithreading-in-php/ >> >> You can use a non-relational database like mongo or couchdb to manage where >> you stop and where you have to look back to the RSS feed as well. >> >> []'s >> >> Regards, >> Igor Escobar >> *Software Engineer >> * >> + http://blog.igorescobar.com >> + http://www.igorescobar.com >> + @igorescobar <http://www.twitter.com/igorescobar> >> >> >> >> >> >> >> On Sat, Sep 10, 2011 at 10:37 PM, Stuart Dallas <stuart@xxxxxxxx> wrote: >> >>> On 10 Sep 2011, at 09:35, muad shibani wrote: >>> >>> > I want to design an application that reads news from RSS sources. >>> > I have about 1000 RSS feed to collect from. >>> > >>> > I also will use Cron jobs every 15 minutes to collect the data. >>> > the question is: Is there a clever way to collect all those feed items >>> > without exhausting the server >>> > any Ideas >>> >>> I designed a job queuing system a while back when I had a similar problem. >>> You can read about it here: http://stut.net/2009/05/29/php-job-queue/. >>> Set that type of system up and add a job for each feed, set to run every 15 >>> minutes. You can then watch the server and tune the number of concurrent job >>> processors so you get the optimum balance between load and speed. >>> >>> -Stuart >>> >>> -- >>> Stuart Dallas >>> 3ft9 Ltd >>> http://3ft9.com/ >>> -- >>> PHP General Mailing List (http://www.php.net/) >>> To unsubscribe, visit: http://www.php.net/unsub.php >>> >>> >> > Thread != Multi Process. -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
