curl question - curl processes not working as the browser does..

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi.

This isn't a Fed isue, but I am using a fed system to run the curls!!!

Not sure if this is appropriate to the group/thread, but I'm flumoxed.


I'm trying to replicate the process for the FF/chrome browser to
generate the data for the final page. The target site is fairfied.edu,
class data.

The basic process, is :
1) get term
2) get subject
3) generate the class data

I've crafted curl (cmdline) statements to generate step 1/2. Step 3 is
where I run into issues.

One can go to the browser, implement firebug (or
livehttpheaders/etc..) to track the cookies/referrer/headers..

The target site/url is:
    https://xessprod.fairfield.edu/FFPRODStudentRegistrationSsb/ssb/term/termSelection?mode=search

The 1st page (term) you select the term in the term select dialog.
This invokes a dynamic process which is the "get" to generate the term
list.

Using the "term" (summer 2017), and then selecting the "submit"
generates the "subject" page. Again, selecting the "subject" dialog
invokes the dynamic process to fill in the sub sel dialog.

Selecting the subj "submit" uses both the term, as well as the subj,
and then generates the assoicated class page.

For the curl:

To generate the termlist:

terms::
curl -sS -A  "Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Firefox/38.0"    --cookie-jar aa.lwp   --cookie aa.lwp   -L
'https://xessprod.fairfield.edu/FFPRODStudentRegistrationSsb/ssb/classSearch/getTerms?searchTerm=&offset=1&max=100&_=1495062777183'

subj::
curl -sS -A  "Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Firefox/38.0"    --cookie-jar aa.lwp   --cookie aa.lwp   -L
'https://xessprod.fairfield.edu/FFPRODStudentRegistrationSsb/ssb/classSearch/get_subject?searchTerm=&term=201705&offset=1&max=100&_=1495062824646'


try for class::
curl -v -sS -A  "Mozilla/5.0 (X11; Linux x86_64; rv:38.0)
Gecko/20100101 Firefox/38.0"    --cookie-jar aa.lwp   --cookie aa.lwp
 -L 'https://xessprod.fairfield.edu/FFPRODStudentRegistrationSsb/ssb/searchResults/searchResults?txt_subject=AC&txt_term=201705&startDatepicker=&endDatepicker=&pageOffset=0&pageMaxSize=10&sortColumn=subjectDescription&sortDirection=asc'


The curl for the class is screwing up and not generating the required
output. As far as I can tell, it seems that it should work, but I'm
missing some step that the browser is doing...

The output for each step/curl is json



In a browser, entering each of the following shoud work. However, the
term/subj urls work, with the class url failing...

terms
https://xessprod.fairfield.edu/FFPRODStudentRegistrationSsb/ssb/classSearch/getTerms?searchTerm=&offset=1&max=10&_=1495062777183

subj
https://xessprod.fairfield.edu/FFPRODStudentRegistrationSsb/ssb/classSearch/get_subject?searchTerm=&term=201705&offset=1&max=10&_=1495063116089

class
https://xessprod.fairfield.edu/FFPRODStudentRegistrationSsb/ssb/searchResults/searchResults?txt_subject=AC&txt_term=201705&startDatepicker=&endDatepicker=&pageOffset=0&pageMaxSize=10&sortColumn=subjectDescription&sortDirection=asc

Any thoughts/comments to point to my "doh" moment!!

thanks


ps, the headers from the browser are::

response header::
Cache-Control    no-cache, no-store
Content-Type    application/json;charset=UTF-8
Date    Wed, 17 May 2017 23:45:09 GMT
Expires    -1
Server    Apache-Coyote/1.1
Set-Cookie    BNI_BARRACUDA_LB_COOKIE=0000000000000000000000000f090d0a0000bb01;
Path=/; HttpOnly
Transfer-Encoding    chunked
X-Frame-Options    DENY
x-ua-compatible    IE=edge

request header::
Accept    application/json, text/javascript, */*; q=0.01
Accept-Encoding    gzip, deflate
Accept-Language    en-US,en;q=0.5
Connection    keep-alive
Cookie    JSESSIONID=6DC2BB4A00F547220A0B1E7AA758D557;BNI_BARRACUDA_LB_COOKIE=0000000000000000000000000f090d0
a0000bb01
Host    xessprod.fairfield.edu
Referer    https://xessprod.fairfield.edu/FFPRODStudentRegistrationSsb/ssb/classSearch/classSearch
User-Agent    Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Firefox/45.0
X-Requested-With    XMLHttpRequest
X-Synchronizer-Token    ae671d70-4f86-4c4f-8d86-4ffe5275de50
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx



[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [EPEL Devel]     [Fedora Magazine]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Desktop]     [Fedora Fonts]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Fedora Sparc]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux