what Robert suggest, is something exactly what the big-big photosgallery out there usedto store their image. Distributing the image into many servers. But iam not very sure, howthey store the image location information inside the database. But i like the idea of using imageUId to determine where to store the image.I think, storing image in a bucket with 100 images each can make the image moremanageable.. thanks for that idea.. On 7/11/08, Robert Cummings <robert@xxxxxxxxxxxxx> wrote:> On Thu, 2008-07-10 at 15:46 +0800, paragasu wrote:>> i am planning to create a web photo gallery. i know there is a lot>> available out there,>> but i really want to create my own. the problem is not about creating>> the photo gallery.>> i want it to be scalable.>>>> the plan is saving the image metadata in the database and keep the>> original files>> in a folder. while it work perfectly, i afraid how many files 1>> directory can keep.>> in 1 year, there going to be more than 1000 photo uploaded or more.>> sure it bring problem>> to maintain thus files.>>>> when i see big2 website out there like friendster, photobucket. the>> url for the image change>> like>> pic-1.domain.com/images/filename.jpg>> pic-2.domain.com/images/filename.jpg>> etc. they seems to put the image in different subdomain. Just wan't to>> ask anyone who>> did create a big-big web gallery like this. How did you do it?>> Those guys are probably having their images hit over a million times a> day. They also likely have millions of images. They are using multiple> servers to load balance the image serving. Whether this also is used to> distribute the images themselves is another question.>> In Linux a single directory can easily handle hundreds of thousands of> files. If you expect there to be million in short time, you can always> bucket the images. Bucketing involves generating some kind of ID that> can be used to generate bucket directories. For instance if you decide> you want 1000 buckets, you can an unique integer ID for images in the> database and modulo the uid by 1000 to determine the bucket directory.> This means if you have 100,000 images then there would be 1000> directories and 100 images in each. As you grow to 1 million images you> would approach 1000 images in 1000 directories. Server offloading can> work similarly. And you can combine approaches. For instance if you have> 10 servers to dedicate to images then you can do the following:>> $serverId = $imageUid % 10;> $bucketId = $imageUid % 1000;>> $url = "http://image{$serverId}.myDomain.com";> ."/{$bucketId}/{$imageFile}";>> So for the first 20 images you would have:>> http://image1.myDomain.com/1/aaaaaa.jpg> http://image2.myDomain.com/2/aaaaab.jpg> http://image3.myDomain.com/3/aaaaac.jpg> http://image4.myDomain.com/4/aaaaad.jpg> http://image5.myDomain.com/5/aaaaae.jpg> http://image6.myDomain.com/6/aaaaaf.jpg> http://image7.myDomain.com/7/aaaaag.jpg> http://image8.myDomain.com/8/aaaaah.jpg> http://image9.myDomain.com/9/aaaaai.jpg> http://image10.myDomain.com/10/aaaaaj.jpg> http://image1.myDomain.com/11/aaaaak.jpg> http://image2.myDomain.com/12/aaaaal.jpg> http://image3.myDomain.com/13/aaaaam.jpg> http://image4.myDomain.com/14/aaaaan.jpg> http://image5.myDomain.com/15/aaaaao.jpg> http://image6.myDomain.com/16/aaaaap.jpg> http://image7.myDomain.com/17/aaaaaq.jpg> http://image8.myDomain.com/18/aaaaar.jpg> http://image9.myDomain.com/19/aaaaas.jpg> http://image10.myDomain.com/20/aaaaat.jpg>> Of course, if you have multiple servers, you need some mechanism on the> backend to migrate uploaded images to the appropriate server.>> Hope that helps.>> Cheers,> Rob.> --> http://www.interjinn.com> Application and Templating Framework for PHP>>
