On 10/22/2016 02:39 PM, bruce wrote:
On Sat, Oct 22, 2016 at 4:00 PM, jd1008 <jd1008@xxxxxxxxx> wrote:
On 10/22/2016 12:14 PM, Gordon Messmer wrote:
On 10/22/2016 06:13 AM, bruce wrote:
I've thought of having a "pid" file on the nfs, where each clientApp
would reset/lock/use the pidFile to then get/use/delete the file as
required, but this seems tb slow.. but doable.
Probably. You'd have a bunch of clients periodically checking for the
lock file before they do work. You'll lose some time to idling when one
application finishes, before another application gets the lock and starts
working. You may be better off having one process scheduling work, starting
up or signaling an application when the data is available. You'll avoid the
complexity of the locking, and possibly reduce idle time.
So, you are suggesting: One process on the nfs server that knows about all
the files in question and waits for clients to ask for a file (any
file????), returns back to that nfs client an nfs handle for some specific
file chosen by the server, creates some flag somewhere (on the server) that
the file has been served to username xyz, uid cde, on client a.b.c. When
is finished with that file, does something like ask the server to delete
that file, or says to the server I am done with that file; at which point
server might delete the file per client request, or simply remove the flag
that the file is in use by username xyz, uid cde, on client a.b.c, ... and
so on.
Well, now you have another wait problem for each request for some unserved
file. In such a serial service of requests - clients might time out, and can
err out or simply retry.
What if the list of files is in the thousands? Serially serving said files
would be very time consuming.
Parallel server processes:
The server process might be coded so that for each request, it forks a child
process to serve the request.
Now, child processes have to compete for a lock on the list of files to be
served, one of the children will succeed, and
set the flag (mark username xyz, uid cde, on client a.b.c has file
/filename/), serve the file, unlock the list and exit. Same client that
requested the file must then request the server to delete the file or tell
the server it is done with the file - again requiring the child server
process (not same child process that served the file, but a new child
spawned to handle the request) to lock the list + delete the file or simply
remove the flag of the client.
On the surface of it, it sounds like this is a slow process. But it is not.
A sever process that spawns children that compete for the list is a good way
to serve said files.
-----
Hey JD!
Thanks for the reply.
I thought about having some sort of "nfs/server" side process that
iterates through the list of files, and then determines when a
"client" is ready, and pushes the file to the client.
In having the clients, pull the file (set the pidFILE), there's the
possibility that the pidFile, could be messed up, along with a few
other minor issues, but it would be simple to implement.
Thanks
-b
Hi Bruce,
Client's .pid files are not shared. They only exist on the client.
So, client A would inevitable end up processing the same file as client B.
I suggest you write a small daemon on the nfs server that does what I
suggest.
The "list" is simply an array of strings (array of pointers to null
terminated character strings.).
This array should be in an include file that also exists on the client
that will build the client application code.
Once built it can be installed on all clients.
the server process will also include this file.
The server process should
1. setup the socket for listening to requests.
2. loop
listen for incoming requests
accept the request (which contains data of the nature of the request)
spawn a thread and pass to it the acceptance socket (the data of
which the thread will process).
endloop
3. The thread will parse the request to see if it is a request for a
file or a request to close a file or a request to delete a file.
The thread will use sychronization primitives to lock the list
mentioned above and perform the request. If the request
is for a file, then it needs to open that file for reading, and
return an nfs handle to the client, do some overhead, like
create the flag of client info (username, uid, client ip), then
unlock the list and return. The parent process will not wait for
the child thread, as it must simply accept connections and spawn
the same thread to handle the request.
Multiply (I do not mean mathematical 'multiply', I mean more than
one), spawned threads are independent of each other.
They only share the the list and the list's lock primitives.
The client will use that handle as a file descriptor and process the file.
When client is done, client will request to a: close the file or b:
close and delete the file.
That's it.
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
