Re: curl - sys env vars in a curl statement

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 08Aug2016 17:13, bruce <badouglas@xxxxxxxxx> wrote:
Got a prob, and maybe I'm just tired. As far as I can tell, sys env vars
should be double quoted >"< inside a curl.

No.

What you have is a shell syntax issue: the quoting of a string affects what the shell will do to it _before_ the command is issued. So:

 x=1
 echo 'x=$x'
 echo "x=$x"

will issue the output:

 x=$x
 x=1

Note that this is NOT DONE BY ECHO, and likewise, not by curl. The shell is constructing the echo command, and for the former it supplies the string

 x=$x

and for the latter the string:

 x=1

So, to your commands:

echo '' > 1aa.lwp

This make a file with a single blank line. You you want a geniunely empty file, use:

 >1aa.lwp

on its own. The shell will open and trucate it, then close it, running no command. Thus, an empty file.

curl -vvv  -A  "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11)
Gecko/2009061118 Fedora/3.0.11-1.fc9 Firefox/3.0.11"   --cookie-jar 1aa.lwp
--cookie 1aa.lwp   -L "
https://public.lionpath.psu.edu/psp/CSPRD/EMPLOYEE/HRMS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL
"

Rule 1: if you are _not_ substituting shell environment values, use single quote instead of doubles. Always. It avoids accidents.

I'm also hoping those isolates quote marks above after the -L are artifacts of the mail cut/paste, and not isolated in your script. If they are, you are supplying curl a string with a leading and trailing newline character. It may be quietly stripping those for you, but the string as supplied should really be nice and clean i.e. _exactly_ the URL you intend, no extra characters.

So: turn these double quotes into single quotes.

icsidval=$(curl -vvv  -A  "Mozilla/5.0 (X11; U; Linux x86_64; en-US;
rv:1.9.0.11) Gecko/2009061118 Fedora/3.0.11-1.fc9 Firefox/3.0.11"
--cookie-jar 1aa.lwp --cookie 1aa.lwp     -e "
https://public.lionpath.psu.edu/psp/CSPRD/EMPLOYEE/HRMS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL";
-L "
https://public.lionpath.psu.edu/psc/CSPRD/EMPLOYEE/HRMS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL";
| grep ICSID | grep -oP "value='\K[^']+")

Same here: single quotes. Especially in your "grep -oP" command: it is merely fortuitous that \K is not special to shell strings. I see you have embedded single quotes in your regexp, perhaps that explains the double quoting. If so, try this:

 grep -oP "value='\\K[^']+"

The shell will reliably turn the \\ into a \ inside the double quotes. As it is the slosh only survives because \K is not special inside shell double quotes. Had you used something special, eg \$ the quote would have been eaten in order to protect the $.

echo $icsidval   ### <<<<< this displays the sys env data..

Excellent.

curl -v  -A  "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.11)
Gecko/2009061118 Fedora/3.0.11-1.fc9 Firefox/3.0.11"   --cookie-jar 1aa.lwp
--cookie 1aa.lwp     -d
'ICAJAX=1&ICNAVTYPEDROPDOWN=0&ICType=Panel&ICElementNum=0&ICStateNum=2&ICAction=SSR_CLSRCH_WRK_CAMPUS%240&ICXPos=0&ICYPos=65&ResponsetoDiffFrame=-1&TargetFrameName=None&FacetPath=None&ICFocus=&ICSaveWarningFilter=0&ICChanged=-1&ICAutoSave=0&ICResubmit=0&ICSID="$icsidval"&ICActionPrompt=false&ICBcDomData=undefined&ICFind=&ICAddCount=&ICAPPCLSDATA=&SSR_CLSRCH_WRK_CAMPUS$0=BK'
-e "
https://public.lionpath.psu.edu/psc/CSPRD/EMPLOYEE/HRMS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL";
-L "
https://public.lionpath.psu.edu/psc/CSPRD/EMPLOYEE/HRMS/c/COMMUNITY_ACCESS.CLASS_SEARCH.GBL
"

Here you're wrongly presuming that the double quotes _inside_ your long single quoted data after the -d "activate" parameter substitution; they do not. Inside single quotes nothing is special except a closing single quote.

What you want to do here, _unlike_ all the other long strings, is to use double quotes around the whole thing:

 "ICAJAX=1&ICNAVTYPEDROPDOWN=0&ICType=Panel&ICElementNum=0&ICStateNum=2&ICAction=SSR_CLSRCH_WRK_CAMPUS%240&ICXPos=0&ICYPos=65&ResponsetoDiffFrame=-1&TargetFrameName=None&FacetPath=None&ICFocus=&ICSaveWarningFilter=0&ICChanged=-1&ICAutoSave=0&ICResubmit=0&ICSID=$icsidval&ICActionPrompt=false&ICBcDomData=undefined&ICFind=&ICAddCount=&ICAPPCLSDATA=&SSR_CLSRCH_WRK_CAMPUS$0=BK"

and get rid of the inner ones - they're doing nothing for you.

BTW, you may find it helpful to turn on shell execution tracing when debugging issues like this. It shows you the commands the shell actually issues, _after_ doing parameter subtitution. So put a:

 set -x

at the start of your script, or at the start of the stuff you're debugging. For this it would have been more helpful to you than curl's -v options.

Cheers,
Cameron Simpson <cs@xxxxxxxxxx>
--
users mailing list
users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe or change subscription options:
https://lists.fedoraproject.org/admin/lists/users@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org



[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [EPEL Devel]     [Fedora Magazine]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Desktop]     [Fedora Fonts]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Fedora Sparc]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux