SVN->GIT link broken?

John Ralls jralls at ceridwen.us
Tue Mar 26 14:24:12 EDT 2013


On Mar 26, 2013, at 10:51 AM, Derek Atkins <derek at ihtfp.com> wrote:

> Hi,
> 
> On Tue, March 26, 2013 12:56 pm, John Ralls wrote:
> 
>> OK. Forward the emails and we can discuss which ones merit handling inside
>> of git-svn-mirror, which can be ignored, and which require human
>> intervention, then set up error handling accordingly. There should
>> probably be a limit on the number of retries which might need to survive
>> from one instance to the next, after which you get a "something's really
>> wrong" email.
>> 
> 
> Here are a few of the errors I saw over the weekend:
> 
> Running git svn fetch
> fatal: Unable to create
> '/u1/home/git-svn/git-svn/gnucash-htdocs/.git/refs/remotes/svn/published.lock':
> File exists.
> 
> If no other git process is currently running, this probably means a
> git process crashed in this repository earlier. Make sure no other git
> process is running and remove the file manually to continue.
> error: failed to run reflog
> gc: command returned error: 255
> 
> ---
> 
> Running git svn fetch
> mv: cannot stat
> `/u1/home/git-svn/git-svn/gnucash-docs/.git/objects/pack/pack-31cc45b7482ee4b7cf3b5e987f6a2927bb2306ce.pack':
> No such file or directory
> error: failed to run repack
> gc: command returned error: 255

I think these two are the result of trying to run two instances. 

> 
> ---
> 
> Running git svn fetch
> remote: ssh: connect to host github.com port 22: Connection timed out     
>  remote:
> remote: fatal: The remote end hung up unexpectedly
> remote: ssh: connect to host github.com port 22: Connection timed out     
>  remote:
> remote: fatal: The remote end hung up unexpectedly
> remote: Update script found and executable

And this is probably why: The timeouts caused the script run time to exceed 1 minute.

> 
> 
>>> 
>>>> BTW, why a cron job instead of an SVN commit hook?
>>> 
>>> In order to change the user so that I didn't have to make sure everyone
>>> with an SVN account could directly write into gitolite..  I wanted to
>>> make sure that the push into gitolite only happened from a single user,
>>> and the best way I could think of to do that was via a cron job.
>> 
>> OK. Note that if git-svn-mirror does a backoff you'll need locking in your
>> cron script. You might consider killing a previous instance if the
>> sentinel file is reset indicating another commit needs processing.
>> 
>> Regards,
>> John Ralls
> 
> Yeah, right now it IS possible for two copies to run if the first run
> takes more than 60 seconds to complete and there were two svn commits
> back-to-back but crossing the cron job execution.  I.e, if you get:
> 
>  svn commit
>  cron execution 1
>  svn commit
>  cron execution 2
>  cron job 1 completes
>  cron job 2 completes
> 
> This situation *is* possible given the current implementation.  One way I
> could fix this would be to make sure cron job 2 cannot start until after
> cron job 1 completes.  Of course I would need to make sure the lock
> disappears if the cron job crashes, to make sure it's not completely
> dependent on a complete run to unlock.
> 
> The cron script is pretty dumb and straightforward:
> 
> #!/bin/bash
> #
> 
> trigger=/home/svn/repo/tmp/trigger
> HOME=${HOME-/home/git-svn}
> 
> # Only run this if we've been tickled
> if [ ! -f $trigger ] ; then
>  exit 0
> fi
> rm -f $trigger
> 
> # Iterate over each repo and sync to gitolite
> #echo "cd $HOME"
> cd $HOME
> for repo in gnucash gnucash-docs gnucash-htdocs ; do
>  ./git-svn-mirror update git-svn/$repo
> done
> 

Here's the "knocker" script I use:
#!/usr/bin/perl -w
use strict;

use Fcntl qw(:DEFAULT :flock);
use Sys::Syslog;
use Tie::Syslog;

$ENV{PATH} = "/usr/local/bin:/usr/bin:/bin";
my $line = <>;
chomp $line;

my $wlock = "/home/john/wait.lock";
my $plock = "/home/john/proc.lock";

my ($waitfd, $lockfd);
close STDIN;
close STDOUT;
close STDERR;
my $logger = tie *STDERR, 'Tie::Syslog', 'local6.info', 'Gnucash_knocker', 'pid', 'unix';
$logger->ExtendedSTDERR();

sysopen($waitfd, $wlock, O_RDWR | O_CREAT)
	or die "Unable to open lockfile $wlock:$!";
my $wait = flock($waitfd, LOCK_EX | LOCK_NB);
if (not $wait) {  #Didn't get the wait lock, someone's already waiting
	close $waitfd;
	exit 0;
}

sleep (2); #Wait a little while for other requests

sysopen($lockfd, $plock, O_RDWR | O_CREAT)
	or die "Unable to open lockfile $plock:$!";
flock($lockfd, LOCK_EX) or die "Failed to acquire the lock: $!";
close $waitfd; #release the wait lock; next request will be queued to wait
print STDERR "Received keyword $line";
my $typere = qr/(gnucash|gnucash-docs|gnucash-htdocs)/;

if ($line && $line =~ /$typere/) {
	$line = "gnucash-trunk" if $line eq "gnucash";
	print STDERR "Processing Directory $line";
	open STATUS, "/home/john/git-svn-mirror update /home/john/$line 2>&1 |";
	while (my $out = <STATUS>) {
		chomp $out;
		print STDERR $out;
	}
}
else {
	foreach my $dir (qw(gnucash-trunk gnucash-docs gnucash-htdocs)) {
		print STDERR "Processing Directory $dir";
		open STATUS, "/home/john/git-svn-mirror update /home/john/$dir 2>&1 |";
		while (my $out = <STATUS>) {
			chomp $out;
			print STDERR $out;
		}
	}
}
close *STATUS;
close $lockfd;
undef $logger;
untie *STDERR;

It's perhaps a bit more involved than what you need because it has to decide which repo to update. You also might prefer to get emails than to use syslog.

Regards,
John Ralls


More information about the gnucash-devel mailing list