On Thu, 2008-07-10 at 15:46 +0800, paragasu wrote:
> i am planning to create a web photo gallery. i know there is a lot
> available out there,
> but i really want to create my own. the problem is not about creating
> the photo gallery.
> i want it to be scalable.
>
> the plan is saving the image metadata in the database and keep the
> original files
> in a folder. while it work perfectly, i afraid how many files 1
> directory can keep.
> in 1 year, there going to be more than 1000 photo uploaded or more.
> sure it bring problem
> to maintain thus files.
>
> when i see big2 website out there like friendster, photobucket. the
> url for the image change
> like
> pic-1.domain.com/images/filename.jpg
> pic-2.domain.com/images/filename.jpg
> etc. they seems to put the image in different subdomain. Just wan't to
> ask anyone who
> did create a big-big web gallery like this. How did you do it?
Those guys are probably having their images hit over a million times a
day. They also likely have millions of images. They are using multiple
servers to load balance the image serving. Whether this also is used to
distribute the images themselves is another question.
In Linux a single directory can easily handle hundreds of thousands of
files. If you expect there to be million in short time, you can always
bucket the images. Bucketing involves generating some kind of ID that
can be used to generate bucket directories. For instance if you decide
you want 1000 buckets, you can an unique integer ID for images in the
database and modulo the uid by 1000 to determine the bucket directory.
This means if you have 100,000 images then there would be 1000
directories and 100 images in each. As you grow to 1 million images you
would approach 1000 images in 1000 directories. Server offloading can
work similarly. And you can combine approaches. For instance if you have
10 servers to dedicate to images then you can do the following:
$serverId = $imageUid % 10;
$bucketId = $imageUid % 1000;
$url = "http://image{$serverId}.myDomain.com";
."/{$bucketId}/{$imageFile}";
So for the first 20 images you would have:
http://image1.myDomain.com/1/aaaaaa.jpg
http://image2.myDomain.com/2/aaaaab.jpg
http://image3.myDomain.com/3/aaaaac.jpg
http://image4.myDomain.com/4/aaaaad.jpg
http://image5.myDomain.com/5/aaaaae.jpg
http://image6.myDomain.com/6/aaaaaf.jpg
http://image7.myDomain.com/7/aaaaag.jpg
http://image8.myDomain.com/8/aaaaah.jpg
http://image9.myDomain.com/9/aaaaai.jpg
http://image10.myDomain.com/10/aaaaaj.jpg
http://image1.myDomain.com/11/aaaaak.jpg
http://image2.myDomain.com/12/aaaaal.jpg
http://image3.myDomain.com/13/aaaaam.jpg
http://image4.myDomain.com/14/aaaaan.jpg
http://image5.myDomain.com/15/aaaaao.jpg
http://image6.myDomain.com/16/aaaaap.jpg
http://image7.myDomain.com/17/aaaaaq.jpg
http://image8.myDomain.com/18/aaaaar.jpg
http://image9.myDomain.com/19/aaaaas.jpg
http://image10.myDomain.com/20/aaaaat.jpg
Of course, if you have multiple servers, you need some mechanism on the
backend to migrate uploaded images to the appropriate server.
Hope that helps.
Cheers,
Rob.
--
http://www.interjinn.com
Application and Templating Framework for PHP
--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php
