Re: Large static files - no mod_xsendfile available

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 8/23/2012 2:55 PM, Knacktus wrote:
> Am 23.08.2012 20:27, schrieb Ben Johnson:
>>
>>
>> On 8/23/2012 1:42 PM, Knacktus wrote:
>>> Hi all,
>>>
>>> I'm a newbie to Apache, so I hope my question makes any sense :-) ...
>>>
>>> I have a Python web application which runs on a RedHat 6.3 Server. I'm
>>> not allowed to install any third party packages or modules, which are
>>> not in the RedHat standard repository.
>>>
>>> My app needs to serve and recieve large static files. The file downloads
>>> need to be authorized. Google pointed me to mod_xsendfile, which was
>>> excatly what I was looking for. Unfortunately, I can't install it on my
>>> server.
>>
>> A bit of a corollary, but I recommend that you switch hosts and purchase
>> a VPS on which you can do anything you like. That said, I realize that
>> this may not be an option for whatever reason (client's server, etc.).
>>
>>> Is there any other way to achieve a similar functionality with the
>>> standard apache modules?
>>>
>>> Thanks,
>>>
>>> Jan
>>
>> While I agree that mod_xsendfile is the ideal approach, I've had no
>> problem sending files up to 2GB in length to the user-agent via PHP
>> script (and Python should be capable of the same).
>>
>> While sending files much larger than 2GB is possible, in theory, there
>> are many obstacles that make doing so a challenge.
>>
>> So, if your files exceed 2GB in length, I see your concern. Otherwise,
>> why not use "straight-up" Apache? If Apache cannot meet your
>> authentication and/or authorization needs adequately, it seems that you
>> have no choice but to use Python for the task, in which case you would
>> also use Python to pass the binary files (from a location outside
>> Apache's document root) to the user-agent.
>>
>> In other words, what, exactly, is the problem?
> 
> First I have to say, that I haven't done any real benchmarks yet. The
> file size is below 1 GB, but the number of related, subsequent requests

(Did you mean "concurrent" requests? 100-500 at the same time?)

> can be around 100 ... 500 files with sizes from about 100 KB to 50 MB.
> Some really large ones are about 200 MB, but those are rare. (The files
> are CAD files.)

In that case, it's safe to say that file length is not an issue in as
much as you will not hit a "ceiling" of any kind. (Performance is
another issue.)

> Right now, I only anticipate a problem because I read about the
> advantages of the x_sendfile module (the files are not read by the
> application code, but transfered directly by Apache). My main concern is
> that the Python processes are blocked by the file uploads.

I'm not sure what you mean by the previous sentence. I thought that we
were talking about downloads (not uploads). Are we talking about both?

If you meant to say file *downloads*, I don't see any reason that you
should encounter a blocking/locking issue. I don't use Python, but with
PHP, any number of Apache requests, from the same user-agent, will be
serviced concurrently. That said, there *are* Apache configuration
directives that affect/limit this behavior (I believe most to be off/set
reasonably by default), and poor session-handling within the application
logic can lead to session-locking (forcing Apache to service only one
request at a time).

> Authorisation is checked per file using some logic and another
> web-service. So I would really like to keep that in my Python app, as
> it's all developed, and I can't even tell, if it's possible at all doing
> it with some Apache magic (newbie ...).

I recommend that you leave the A/A in Python, too. While it may be
possible to achieve the same effect in Apache, there's no reason to
change your existing logic.

> Probably I have to go the direction you pointed out (passing the file
> via Python). Would it be possible / advisable to use two distinct
> applications with two Apaches on different ports?

I don't know anything about Python, so I can't say with certainty, but
it seems that you may run into problems with user sessions and your
authentication scheme if you attempt that. Further, I'm not sure how
much you will realize in the way of performance gains, if any.

We would have more to go on if you actually went ahead with a basic
attempt and let us know of any perceived issues.

At this point, it's clear that mod_xsendfile is not an option for you,
so you have no choice but to put one foot in front of the other. :)

> One for the files, the
> other for meta-data? I have full control over the client, which is a
> richt-client. So I can route the requests for static files to one port,
> the other requests to another port.
> 
> Cheers,
> 
> Jan
> 
> 
>>
>> Thanks,
>>
>> -Ben


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx



[Index of Archives]     [Open SSH Users]     [Linux ACPI]     [Linux Kernel]     [Linux Laptop]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Squid]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]

  Powered by Linux