Re: Fastest lookup mechanism

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



[Note that I changed the thread title to reflect that this isn't really 
about templating.]

> Yes I thought of this, but in my case a flat file would be better. The 
> same problem applies though:
> [quote]
> This is all no problem, except that these lists can be pretty big. And I
> wouldn't like to load them all into the memory, or have to loop through
> the file every time translate is called.
> [/quote]

I don't understand your comments.  Why would a flat file be better?  
Unless you have fixed-length lines and a sorted file so that you can do 
a binary search, a database is going to be several orders of magnitude 
faster than a flat file.

Look, fundamentally you have a simple problem -- to find a needle in a 
very well-defined, orderly haystack.  This is a problem that many 
people have solved in the past and for which very robust solutions 
exist.  Those solutions are things like tree search, binary search, and 
hash table lookups.  An indexed database might use any of these, arrays 
in memory use hashing.

So these solutions exist but it seems you are saying "I cannot use any 
of those, I need to use a flat file, now tell me how to make it fast 
enough to be able to do perhaps hundreds of lookups per page".  My 
guess is you simply can't meet that goal with nothing but a flat file, 
unless your standards for page display time are very low.

The best way to make it faster is to convert your flat file to a 
database and/or load it as an array.  So my first question is, *why* 
are you objecting to these tried and true methods of answering exactly 
the question you are asking?

If you must use a pure flat file, the fastest search method I know of 
is to maintain it in sorted order with fixed-length lines and then do a 
binary search.  Algorithms for this are readily available if you are 
not familiar with the technique [a great if expensive source for 
algorithms of this type is the classic book "The Art of Computer 
Programming", volume 3, Searching and Sorting by Donald E. Knuth].  But 
binary search on a large flat file is still likely to be unacceptably 
slow for hundreds of lookups per page.

Caching will help but you will have the exact same problem with the 
cache -- how do you find the data you want within the cache?

--
Tom

-- 
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


[Index of Archives]     [PHP Home]     [Apache Users]     [PHP on Windows]     [Kernel Newbies]     [PHP Install]     [PHP Classes]     [Pear]     [Postgresql]     [Postgresql PHP]     [PHP on Windows]     [PHP Database Programming]     [PHP SOAP]

  Powered by Linux