aha.. we have success... steps to repeat: generate the base page to get the initial content, as well as the cookie wget -S "https://isiscc.smc.edu/pls/apex/f?p=123:1:12101780574999:::::"; -O smc.dat extract the required cookie: ORA_WWV_APP_123=ORA_WWV-HtXgmY8ZHOxc5O9zc8N8j6tP >>>>ORA_WWV-HtXgmY8ZHOxc5O9zc8N8j6tP<<< this goes in the wget as the cookie header use the returned smc.dat file to determine the required attributes that change. In this case, we need &p_instance=257747591540 &p_page_submission_id=4686527047365 &p_page_checksum=DB5BF68CB0895706468BB180F90F4C3B search/copy these vals from the data in order to generate the post data, you can do it in a couple of ways. I viewed the returned post data from a few different pages of the site in my fav viewer/debugger, and determined that all of the variables appear to be the same/consistent. For real accuracy, and in a crawler, it would be useful to simply do a xpath on the hidden name/vals and use that data to generate the post-data. but we cheat for now: so the base post-data consists of p_flow_id=123&p_flow_step_id=1&p_request=SUBMIT&p_arg_names=6371933991931405031&p_t01=Spring+2015&p_arg_names=6362920808091113318&p_t02=1+%2C+2+%2C+3+%2C+4+%2C+5+%2C+6+%2C+7+%2C+12&p_arg_names=6362811294857854001&p_t03=%27OPEN%27+%2C+%27CLOSED%27&p_arg_names=6374897696943587288&p_t04=20151&p_arg_names=3851224476948135099&p_t05=1&p_arg_names=3851225084220137224&p_t06=2&p_arg_names=6361666501512343398&p_arg_names=6362829299321902607&p_arg_names=6363404907668258054&p_arg_names=6363528795332614067&p_arg_names=6363575091232717019&p_arg_names=6363649096707888902&p_arg_names=3851458085236906098&p_t13=April+++++02%2C+2015+%40+12%3A05+pm&p_arg_names=6372068906550664708&p_t14=cls_web_list_20151&p_arg_names=6372277789412047762&p_t15=web_cat_sched_20151&p_arg_names=6361721707633487104&p_t16=where+cat_grp_seq_num+in+%28cat_grp_seq_num%29+and+cls_status+in+%28%27OPEN%27+%2C+%27CLOSED%27%29+and+print_seq+in+%28print_seq%29+and+print_seq+in+%281+%2C+2+%2C+3+%2C+4+%2C+5+%2C+6+%2C+7+%2C+12%29+and+nvl%28gnr_emply_name%2C%27*%27%29+in+%28nvl%28gnr_emply_name%2C%27*%27%29%29+and+nvl%28ssn_days%2C%27*%27%29+in+%28nvl%28ssn_days%2C%27*%27%29%29+and+sct_begin_week+in+%28sct_begin_week%29+and+nvl%28begin_time%2C%270%27%29+in+%28nvl%28begin_time%2C%270%27%29%29&p_md5_checksum=&p_instance=257747591540&p_page_submission_id=4686527047365&p_page_checksum=DB5BF68CB0895706468BB180F90F4C3B&p_v07=80 the class ID p_v07 changes based on the sel/opt within the page -80 happens to be for chem. &p_v07=80 so combining everything together... we get the wget wget --no-cookies --header "Cookie: ORA_WWV_APP_123=ORA_WWV-HtXgmY8ZHOxc5O9zc8N8j6tP" --user-agent="Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11) Gecko/2009061118 Fedora/3.0.11-1.fc9 Firefox/3.0.11" -O - --post-data="p_flow_id=123&p_flow_step_id=1&p_request=SUBMIT&p_arg_names=6371933991931405031&p_t01=Spring+2015&p_arg_names=6362920808091113318&p_t02=1+%2C+2+%2C+3+%2C+4+%2C+5+%2C+6+%2C+7+%2C+12&p_arg_names=6362811294857854001&p_t03=%27OPEN%27+%2C+%27CLOSED%27&p_arg_names=6374897696943587288&p_t04=20151&p_arg_names=3851224476948135099&p_t05=1&p_arg_names=3851225084220137224&p_t06=2&p_arg_names=6361666501512343398&p_arg_names=6362829299321902607&p_arg_names=6363404907668258054&p_arg_names=6363528795332614067&p_arg_names=6363575091232717019&p_arg_names=6363649096707888902&p_arg_names=3851458085236906098&p_t13=April+++++02%2C+2015+%40+12%3A05+pm&p_arg_names=6372068906550664708&p_t14=cls_web_list_20151&p_arg_names=6372277789412047762&p_t15=web_cat_sched_20151&p_arg_names=6361721707633487104&p_t16=where+cat_grp_seq_num+in+%28cat_grp_seq_num%29+and+cls_status+in+%28%27OPEN%27+%2C+%27CLOSED%27%29+and+print_seq+in+%28print_seq%29+and+print_seq+in+%281+%2C+2+%2C+3+%2C+4+%2C+5+%2C+6+%2C+7+%2C+12%29+and+nvl%28gnr_emply_name%2C%27*%27%29+in+%28nvl%28gnr_emply_name%2C%27*%27%29%29+and+nvl%28ssn_days%2C%27*%27%29+in+%28nvl%28ssn_days%2C%27*%27%29%29+and+sct_begin_week+in+%28sct_begin_week%29+and+nvl%28begin_time%2C%270%27%29+in+%28nvl%28begin_time%2C%270%27%29%29&p_md5_checksum=&p_instance=257747591540&p_page_submission_id=4686527047365&p_page_checksum=DB5BF68CB0895706468BB180F90F4C3B&p_v07=80" -L "https://isiscc.smc.edu/pls/apex/wwv_flow.accept"; now, it might be useful to test to see if this will also work against the base initial page of wget -S "https://isiscc.smc.edu/pls/apex/f?p=123:1:0::NO:RP:"; -O smc.dat I suspect it should thanks for your help on this.. I'm reposting back to the fed list in case anyone needs something like this in the future! I was never able to get this working for the curl/pycurl process. It appears to have something to do with the cert/ssl issues... thanks On Thu, Apr 2, 2015 at 4:49 PM, Alchemist <raimiiic@xxxxxxxxx> wrote: > > > 2015-04-02 23:30 GMT+03:00 bruce <badouglas@xxxxxxxxx>: >> >> Hmm.. >> >> ok..here;s the process I'm going to test... (and by the way. thanks >> much for the cycles you've used to help with this. Helps to sometimes >> have someone to bounce things off of! much appreciation) >> >> it appears that we can generate the "cookies" (or at least you can) >> from the initial 2 line cookie script/test - and that this can then be >> inserted in the wget, using "post data" >> >> So the issue then appears to be how to generate the required post data. >> >> If we assume the site generates key attributes in the base/top level >> page, and that these attributes can then be used in the post-data for >> the subsequent calls, it becomes a matter of then determining which >> attributes are required to be used from the base page (as it changes), >> and which attributes can simply be "set" and not changed. >> >> I'm going to test this by taking the base page, ripping off the xpath >> to get the name/val data, and to see if this works.. >> >> By the way, with the updated data you replied with, just how did you >> regenerate the post data? >> >> thanks >> > > Yes. A quick look uncovered, that wwvFlowForm contains some (perhaps all of > required to send valid request) of theese attributes. Also take a look at > pPageChecksum, and apex_layout_***longint*** in html code. > > JS responsible to generate request is located here > https://isiscc.smc.edu/i/libraries/apex/minified/legacy.min.js > > and some other here > https://isiscc.smc.edu/i/libraries/apex/minified/desktop_all.min.js > https://isiscc.smc.edu/i/libraries/apex/minified/widget.checkboxAndRadio.min.js > > It's captured resquest from site generated request, just to show differences > in POST fields. Yes and both (ORA_WWV-nFa8PnpRSIhvaZ1y372N5oNh and still > working for me ORA_WWV_APP_123=ORA_WWV-GZ16jkW5CQEUTFD0Qi9Xis6Y) -- users mailing list users@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
