Excerpts from C Anthony Risinger's message of 2011-07-21 23:42:21 +0200: > i wanted to share a couple of loose thoughts on the standard practices > for PKGBUILDs ... specifically development ones ... with an immediate > focus on git ones (for this instance, but applies to any) > > awhile back i tried to make a super cheap way of doing git checkouts for builds: > > https://bbs.archlinux.org/viewtopic.php?id=86366 > (and this would have worked if i would have thought to make git use > itself as an alternative object pool ...) > > ... the motivation of this message is to encourage adding some > routines to the makepkg library, similar to msg/msg2/warning/etc, to > handle development checkouts. basically i see people using _gitroot > and _gitname for different stuff ... namely _gitname. i'd like to see > a routine that pulls the repo to a known location so it doenst get > blown away every update, and so the user doesnt have to manually > manage this. everyone does it differently, puts the repository in all > sorts of different locations/etc, and it just isnt very pretty IMO. > > ... so, i suggest a permanent setup based off my out setup and > experiences, in reference of this PKGBUILD in particular: > > http://aur.archlinux.org/packages/pyjamas-engine-pythonwebkit/PKGBUILD > > ) allow _git* variables to be altered by the environment > ) add _gitspec variable which supercedes _gitname for checkout stage > (so you can do relative checkouts or SHA1 based checkouts) > > ... in the routine ... > > ) use a targeted fetch command instead of a blind clone. not only is > this more flexible, but it can significantly reduce download size (for > kernel its a ~50% reduction IIRC). the methoud from my builds are > very simple, greatly resembling --mirror mode in git, but ONLY for the > exact branch you are building > ) store repositories in a known list of locations. my PKGBUILD searches these: > > /var/abs/PKGBUILD.devel/${pkgname}.git > /var/abs/local/PKGBUILD.devel/${pkgname}.git > ${SRCDEST}/PKGBUILD.devel/${pkgname}.git > ${startdir}/PKGBUILD.devel/${pkgname}.git > ~/PKGBUILD.devel/${pkgname}.git > > ... and chooses the first one that already exists (PKGBUILD.devel > location only BEFORE appending repo name), OR the first one with write > access to parent dir (it will create PKGBUILD.devel for you if it > can). i personally used the first one (/var/abs/PKGBUILD.devel), with > `chmod 1777` to sticky bit it like /tmp. this not only allows for > reuse, but also allows for safely+easily+consistently bind-mounting > into a chroot for use with mkchrootpkg ... > > ) use a proxy mechanism in the event a repository is found but it is > read-only. this lets you read-only bind mount a repo (think > mkchrootpkg), and it will simply create a new repository, copy the > refs, and setup the object directory as an alternative for the proxy > repo. thus the proxy has the ability to download new objects as > needed, but starts from the same spot as the bind-mounted repo. > > these techniques save me a great deal of time and pain. that package > in particular is a custom webkit build, requiring over 1GB to clone > ... mkchrootpkg tried to blow it away once, bit it was read-only :-) > ... i intend to possibly improve on this process by reducing a > targeted fetch to a *shallow* targeted fetch , ie. the minimum amount > of objects required to build. i have seen "clones" go down to 50%, > sometimes even 10% or less, by using this over a naive `git clone > $xyz`. > > i'd like to see *something* adopted so we can end the madness :-) ... > below are a couple key excerpts from the noted PKGBUILD, for your > reference. > > C Anthony > > ------------------------------------------------- > [locate repo] > ------------------------------------------------- > > # Devel directory fragment > : ${_dir_devel:=PKGBUILD.devel} > > # Local/custom repo? > if [[ -z ${_gitrepo} ]]; then > search=(/var/abs{,/local}/"${_dir_devel}" > "${SRCDEST%%/}/${_dir_devel}" > "${startdir%%/}/${_dir_devel}" > ~/"${_dir_devel}") > for d in "${search[@]%%/}"; do > if [[ -e ${d} ]] || [[ ! -e ${d} && -w ${d%/*} ]]; then > mkdir -p "${d}" 2>&- && > _gitrepo_proxy="${srcdir}/${pkgname}.git" && > _gitrepo="${d}/${pkgname}.git" && > break > fi > done > fi > > ------------------------------------------------- > [create new repo, or new proxy, if needed] > ------------------------------------------------- > > mkdir -p "${w}" > if [[ ! -e ${g}/objects ]]; then > msg "[git] Creating NEW repository ... " > git --git-dir="${g}" --work-tree="${w}" init > elif [[ ! -w ${g}/objects ]]; then > warning "[git] Repository read-only, setting up proxy ... " > git --git-dir="${_gitrepo_proxy}" --work-tree="${w}" init > echo "${g}/objects" > "${_gitrepo_proxy}/objects/info/alternates" > cp -r "${g}/refs" "${_gitrepo_proxy}" > g="${_gitrepo_proxy}" > fi > > ------------------------------------------------- > [perform targeted fetch] > ------------------------------------------------- > > msg "[git] Syncing ... ${_gitroot} -> ${g}" > git --git-dir="${g}" --work-tree="${w}" fetch -fu "${_gitroot}" > "+${_gitname}:${_gitname}" > msg "[git] Reading ... ${_gitspec:-${_gitname}} -> ${w}" > git --git-dir="${g}" --work-tree="${w}" read-tree --reset -u > "${_gitspec:-${_gitname}}" I'm not quite sure what problem you are trying to solve but what I see is that this is an order of magnitude more complex than the current PKGBUILD-git.proto. As it stands here I'd have no idea how to use it, what it does or how it works. I wouldn't want to use it. Just 2c from someone who is neither experienced git user nor fond of bash.
