|
In order to process a large amount of data I need to run a procedure using parallel batch processes.
The query I run in each process is expected to ... 1- select a bunch of id (500) in a table (wait4processing) containing the list of all records to process 2- remove selected records from wait4processing table in order to eliminate duplicate processing 3- run the procedure (build_contributions_clusters) over the range of selected ids --The query I use: With ids as( delete from wait4processing where id in( select id from wait4processing limit 500) returning id) select build_contributions_clusters(min(id),max(id)) from ids; The query runs properly if I send it sequentially (wait for the completion of the query before sening it again) but it does'nt work when sending multiple occurrences in parallel. Seems from the results I got that the first query received by the server runs properly but the following ones try to process the same first 500 records even if deleted - the build_contributions_clusters procedure receive NULL values instead of the following 500 records. Since I am almost certain it is the expected behavior, I would like to like to understand why, and I would also appreciate to see alternative queries to do the job. Thanks :-) |
