Rambling around foo

I have decided that is high time to declare svn-buildpackage orphaned since many people are replying on it to be useful, but nobody cared enough to answer to the RFH Eduard Bloch made some 3 years ago.

I know it would have been better if I would have asked a request for adoption, but, honestly, the package didn't had a maintainer in the last year and that would have been lying to everybody.

The package is quite important and I would hate to see it leave the official archive since many people do need it and use it regularly (I know for sure the GNOME, Perl and the Debian Games Team use it) and I have enjoyed working on the package while I was motivated.

The code is maintained in SVN and the trunk can be viewed at:

http://svn.debian.org/wsvn/collab-maint/deb-maint/svn-buildpackage/trunk/

and can be checked out with any of these commands:

# read-only copy
svn co svn://svn.debian.org/svn/collab-maint/deb-maint/svn-buildpackage/trunk svn-buildpackage

# read write, if you have an account on alioth.debian.org:

svn co svn+ssh://svn.debian.org/svn/collab-maint/deb-maint/svn-buildpackage/trunk


Also, note that the package has several features which the new maintainer(s) should be aware of and must make sure they all work when changing the code. Also, there are some things to remember:

• there is support for native and non-native packages
• non-native packages' repo can be a complete repo (with an upstream branch) or an incomplete repo (just the differences from the upstream - most times the debian directory, but not always)
• there is an interactive and a non-interactive mode, both have to work properly
• the packages can be stored in the repo under 2 different layouts and both have to work since they are both used; support for an arbitary layout would be great and I have a few ideas about how to implement that, so please contact me if you want to work on that
• team maintainance workflow MUST be supprted and should be one of the highest concerns for the maintainer; this means that being able to work on a new package without issuing more than one command to have all necessary stuff for the build, package upgrades to be done is the target to reach for (currently the situation is a lot better than it was in the past)
• there is on misfeature from the past that should be finally dealt with - something that was supposed to work as a cache ended up working like an override; meanwhile the effect for new packages/checkout has been minimised, but the code could do better at dealing with old checkouts affected but this (drop the old file name for the overrides, use a new one and scream when the old file name is encountered)
• 0.6.24 is waiting for an upload
• a test frame is missing and would be most welcome; I already started working on this based on the git test frame, but didn't managed to have a good-enough-to-publish code
  

These being said, I am really hoping svn-buildpackage will find a new maintainer since it deserves a lot more attention than I am providing. Don't forget, 0.6.24 is wating for an upload already.

---

Update: Neil Williams answered and agreed to take over maintenance, but he doesn't usually use svn-inject and svn-upgrade, so somebody taking care of those would be needed. Ironically, those two scripts are the ones in the worst shape of the three main scripts.

I have been thinking for quite some time about migrating the maintenance of the wormux package to git, but in a really smart way so that:

• we preserve history currently stored in svn
• a new integration branch should exist - we used the incomplete package model with svn-buildpackage, but now it makes more sense to have the complete history, including the upstream code at the time
  
• all previous orig tarballs should exist in the git repo in pristine-tar format
• upstream's svn repo should be included entirely so that we can cherry-pick any patches we might want or need

Since both the package repo and the upstream repo are in SVN, you can't simply use git-svn for both in a single repo, so I started thinking of some scheme to be able to join both repos and the (uncommitted) orig tarballs (which are backed up on DGT's webspace).


Tonight I have experimented a little with my git-svn clones which I made previously and eventually thought that:

• the main repo should track the upstream repo via git-svn
• I should properly tag the debian package versions in the svn.d.o repo, so svn tags no longer look like branches in git
  
• ideally all the branches in the package repo should be reproduced in some corresponding branch in the final repo
• the integration branch (that should contain the real debian complete packages) should be created with a non-existant merge strategy which I call overlay (basically, for two branches A and B, the strategy resumes to: all files as in A are present, any files unique to B are present)
• using pristine-tar forces me to keep the 'upstream' branch reserved so I need to avoid using that

It seems the last point could be obtained in some way via this small set of commands (I am sure it needs some modification to make the integration work in another branch C):

git merge --no-commit -s ours B
git checkout B -- .
git checkout A -- .
git commit


Note: Looking back, I think I can do a better job next time and I will se if I can use the resulting repo to create a new repo and drop the svn package maintainance for wormux. If this works, I might be able to create a script to allow migration of svn-bp maintained packages to git, at least for the Debian Games Team.


Since you can't fix a problem if you don't take things one at a time, I started by cloning the initial git-svn generated repos.

So after waiting some time (both git-svn fetch commands took long enough), I had to git-svn repos:

• repo A - the package repo
• repo B - the upstream svn clone repo

I tried to clone both A and B, to avoid distroying the original A and B clones and noticed that the svn branches were not visible from the cloned repo since the origins didn't had any other local branches than master - the svn trunk for each of them. So I added tracking branches in A for the experimental and sarge versions of the package. I also added some tags in A so that the svn tags don't get lost in between.

In the end I got my A' repo that had:

0 eddy@heidi ~/usr/src/wormux/tt/debian-bare $ git br -a
debian-experimental
debian-sarge
debian-sid
* master
origin/debian-experimental
origin/debian-sarge
origin/master

Yes, and having master in A' is not a good idea, but since I made A' a bare repo I had to suffer. Well, one more thing to better in iteration 2.


Excellent, now I can go in B and add the new remote repo which is A' which I should have called pkgsvn, but I ended up call it 'origin'. Since git-svn doesn't treat the svn repo as a true remote, this was possible.



Due to some confusion I also added 'ups-' as a prefix to the svn remote trunk and tags, but I am still unsure if this was necessary - I'll have to see more clearly into the pristine-tar restrictions area.

After all of this I got this branch configuration:
0 eddy@heidi ~/usr/src/wormux/tt/git-full $ git branch
* master
0 eddy@heidi ~/usr/src/wormux/tt/git-full $ git branch -r
0.8-final
0.8beta4
origin/debian-experimental
origin/debian-sarge
origin/debian-sid
origin/master
ups-tags/wormux-0.7
ups-tags/wormux-0.7.2
ups-tags/wormux-0.7.3
ups-tags/wormux-0.7.4
ups-tags/wormux-0.7.9
ups-tags/wormux-0.7.9rc1
ups-tags/wormux-0.7beta3
ups-tags/wormux-0.8
ups-tags/wormux-0.8alpha1
ups-tags/wormux-0.8beta1
ups-tags/wormux-0.8beta2
ups-tags/wormux-0.8beta3
ups-tags/wormux-0.8beta4
ups-trunk
wormux-0.7
wormux-0.7.9
wormux-0.8beta1
wormux-0.8beta2

So more tracking branches were necessary; I fired 'gitk --all' and got a nice picture, after adding tracking branches for branches originating from A':


This is not yet complete and now I am feeling sleepy, so I'll continue once I experiment more.

JoeyH wrote a while back some thoughts on why he hates the DGT SVN repo (mainly our policy to import just the incomplete source of packages - usually just the debian/ directory).

He updated that blog entry and latter said:

Update: I realized after posting this that while the space gains on alioth might be illusory, there is certianly a space gain for people checking the whole repo out. What's really needed is a way to keep all the upstream sources in svn, but check them out only when wanted, and check out all the debian directories for cross-game work.

I was thinking, how about having a different area of the repo where the full source is present and which pulls the debian directory as an external from the current location?

I know it has the huge disadvantage that anything outside of the debian/ directory, but part of the diff, is, for sure, lost. (Still, there is also another team policy to not track anything outside debian directly, except through patches[1], except for really exceptional cases, but even then, there is a need for a strong justification).


Would that work? (I know git-svn doesn't support externals properly, yet, so there's not much gain).


[1] I know Joey, you hate those, too

After a really crazy weekend in which it looked like I wasn't able to set up NAT[1], I went to work with the gear and tested the exact same thing I did at home and it worked without a glitch.

So, now I suspect that there might be something wrong with my ISP or the DSL modem, while ritter (my NSLU2) is fine. As a bonus I just installed Debian armel on it (installation report to arrive soon).


When I got home, after the laptop came back from hibernate[2] I saw these messages on all my terminals:

Message from syslogd@bounty at Mon Nov 12 23:57:04 2007 ...
bounty kernel: Uhhuh. NMI received for unknown reason b0.

Message from syslogd@bounty at Mon Nov 12 23:57:04 2007 ...
bounty kernel: You have some hardware problem, likely on the PCI bus.

Message from syslogd@bounty at Mon Nov 12 23:57:04 2007 ...
bounty kernel: Dazed and confused, but trying to continue

Is this a good time to panic?
I guess I'll have to dig into this, too.


In other news, svn-buildpackage 0.6.23 has been uploaded to unstable and it fixes yet another 7 bugs, which brings svn-bp's bug count down to 19 open valid bugs (2 more if you count wontfix bugs, too). This is the lowest bug count of svn-buildpackage since at least the end of April, according to the bug count graph.

Also, oolite 1.65-6 was also uploaded to unstable and fixes the breakage due to the gnustep build tools changes. Unfortunately, on arm is dep-waiting for libgnustep-base-dev which is broken on this arch.


Thanks to my sponsors, you know who you are ;-) .

[1] I have been trying to set up a simple NAT machine for more than 6 hours and, although everything seemed to be OK, checked and double checked it didn't work
[2] which works now only if I don't use fglrx which is broken from this PoV (bug 449095)

I've been really busy lately in RL and Debian work took the hit.

Some news:

• on the 1st of November, the compendiums on i18n.debian.net managed to waste a huge chunk of disk space and made other scripts (and itself) on churro fail miserably; now there is only a 7 days backlog of the compendiums
• oolite needs an upload due to the gnustep libs transition; I'll try to prepare tonight the long due 1.65-6 version for unstable
• wormux upstream is preparing for yet another beta which should be really close to the final 0.8 version; I wish I had more time to work on this game
• sadly, no news on the naughtysvn front :-( from me
• I have been coding now and then on svn-buildpackage 0.6.23 and I intent to make yet another drop in the bug count visible on the graph:

Thanks to Damyan Ivanov for the upload, svn-buildpackage 0.6.22 is now in experimental. The upload was done to experimental due to the big number of changes affecting it and because I wanted to get a fairly significant amount of testing of the major fixes before propagating the code to unstable.

This release should fix 15 (yes, fifteen) bugs[0], most of which were important bugs or bugs affecting usability.

The major fixes are:

• mkdir -p like functionality in the repo for the tags and other possibly missing directories - this means that even repositories created with older broken versions should be fixed automatically[1]
  
• .svn/deb-layout is no longer a broken cache, but only a real local override for the layout information; .svn/deb-layout is created only on express request via --svn-savecfg; note that although .svn/deb-layout is no longer created automatically, old checkouts should be purged of this cruft (unless the override is wanted)
  
• build dependencies are not required on --svn-export
• automatic creation of the origDir when using origUrl
• some code clean up
  

Installing the package on an etch, lenny or sid system should be straight forward: just get the deb from experimental and install it (no backporting is necessary).


If you usually use svn-buildpackage, please install the experimental version and report any bugs and/or success stories. I am particularly interested in feedback related to the behaviour around .svn/deb-layout and the other methods of specifying layout information.

Please send success stories to me directly eddy.petrisor @ gmail.com. Bugs should be directed to the BTS, but I hope there will be mail just directly to me :-) .


[0] I wonder, does the thickness of the yellow area on these graphs ever decreases?
[1] as soon as the breakage would have been visible in the past

Another weekend trip to my parents' meant another successful svn-buildpackage coding session.

I worked on expanding the svnMkdirP functionality to svn-buildpackage and checked svn-upgrade. So when the testing is done (don't know when that will happen :-( ) I'll merge the work done in the svnmkdir-p branch[1] into trunk and ask for a sponsor - I don't think that will be a problem :-P.


If you want to test, don't try trunk yet[*], your should try the svnmkdir-p branch[1]. Don't forget to bump down the version.


BTW, svn-buildpackage 0.6.22 should close .....


0 eddy@bounty ~/usr/src/svn-buildpackage/svnmkdir-p $ dpkg-parsechangelog | grep ^Closes | sed -r -e 's#Closes:##' -e 's#[0-9]{6}#1 +#g' -e 's#\+$##' | bc
13

13 bugs.

Here are the bastards:

1. 408690
2. 411666
3. 414581
4. 419996
5. 423487
6. 428225
7. 428689
8. 433404
9. 433536
10. 434932
11. 435746
12. 436133
13. 436554

[1] svn+ssh://svn.debian.org/svn/collab-maint/deb-maint/svn-buildpackage/branches/svnmkdir-p or svn://svn.debian.org/svn/collab-maint/deb-maint/svn-buildpackage/branches/svnmkdir-p

[*] the branch is not yet merged into trunk because it needs more thorough testing