Merge branch 'fl/cvsserver'

* fl/cvsserver:
  config.txt: Add gitcvs.db* variables
  cvsserver: Document the GIT branches -> CVS modules mapping more prominently
  cvsserver: Reword documentation on necessity of write access
  cvsserver: Allow to "add" a removed file
  cvsserver: Add asciidoc documentation for new database backend configuration
  cvsserver: Corrections to the database backend configuration
  cvsserver: Use DBI->table_info instead of DBI->tables
  cvsserver: Abort if connect to database fails
  cvsserver: Make the database backend configurable
  cvsserver: Allow to override the configuration per access method
  cvsserver: Handle three part keys in git config correctly
  cvsserver: Introduce new state variable 'method'

Conflicts:

	Documentation/config.txt

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 7e41ca6a0d3e4329d1efc10b4dcb90613975e5bd..2c0a66632329dae8ae1bf9412a57ab247cc8a603 100644 (file)
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -423,8 +423,34 @@ gitcvs.allbinary::
        causes the client to treat all files as binary files which suppresses
        any newline munging it otherwise might do. A work-around for the
        fact that there is no way yet to set single files to mode '-kb'.
+
+gitcvs.dbname::
+       Database used by git-cvsserver to cache revision information
+       derived from the git repository. The exact meaning depends on the
+       used database driver, for SQLite (which is the default driver) this
+       is a filename. Supports variable substitution (see
+       gitlink:git-cvsserver[1] for details). May not contain semicolons (`;`).
+       Default: '%Ggitcvs.%m.sqlite'
+
+gitcvs.dbdriver::
+       Used Perl DBI driver. You can specify any available driver
+        for this here, but it might not work. git-cvsserver is tested
+       with 'DBD::SQLite', reported to work with 'DBD::Pg', and
+       reported *not* to work with 'DBD::mysql'. Experimental feature.
+       May not contain double colons (`:`). Default: 'SQLite'.
        See gitlink:git-cvsserver[1].
 
+gitcvs.dbuser, gitcvs.dbpass::
+       Database user and password. Only useful if setting 'gitcvs.dbdriver',
+       since SQLite has no concept of database users and/or passwords.
+       'gitcvs.dbuser' supports variable substitution (see
+       gitlink:git-cvsserver[1] for details).
+
+All gitcvs variables except for 'gitcvs.allbinary' can also specifed
+as 'gitcvs.<access_method>.<varname>' (where 'access_method' is one
+of "ext" and "pserver") to make them apply only for the given access
+method.
+
 http.sslVerify::
        Whether to verify the SSL certificate when fetching or pushing
        over HTTPS. Can be overridden by the 'GIT_SSL_NO_VERIFY' environment

diff --git a/Documentation/git-cvsserver.txt b/Documentation/git-cvsserver.txt
index f9e0c7737952891633a1f5503f8dc5ad46fbf53f..d22844ba49859b9a189317744e0f14431267e60a 100644 (file)
--- a/Documentation/git-cvsserver.txt
+++ b/Documentation/git-cvsserver.txt
@@ -31,6 +31,10 @@ over pserver for anonymous CVS access.
 
 CVS clients cannot tag, branch or perform GIT merges.
 
+git-cvsserver maps GIT branches to CVS modules. This is very different
+from what most CVS users would expect since in CVS modules usually represent
+one or more directories.
+
 INSTALLATION
 ------------
 
@@ -65,9 +69,22 @@ env variable, you can rename git-cvsserver to cvs.
 
 ------
 Note: you need to ensure each user that is going to invoke git-cvsserver has
-write access to the log file and to the git repository. When offering anon
-access via pserver, this means that the nobody user should have write access
-to at least the sqlite database at the root of the repository.
+write access to the log file and to the database (see
+<<dbbackend,Database Backend>>. If you want to offer write access over
+SSH, the users of course also need write access to the git repository itself.
+
+[[configaccessmethod]]
+All configuration variables can also be overriden for a specific method of
+access. Valid method names are "ext" (for SSH access) and "pserver". The
+following example configuration would disable pserver access while still
+allowing access over SSH.
+------
+   [gitcvs]
+        enabled=0
+
+   [gitcvs "ext"]
+        enabled=1
+------
 --
 3. On the client machine you need to set the following variables.
    CVSROOT should be set as per normal, but the directory should point at the
@@ -93,6 +110,90 @@ Example:
      cvs co -d project-master master
 ------
 
+[[dbbackend]]
+Database Backend
+----------------
+
+git-cvsserver uses one database per git head (i.e. CVS module) to
+store information about the repository for faster access. The
+database doesn't contain any persitent data and can be completly
+regenerated from the git repository at any time. The database
+needs to be updated (i.e. written to) after every commit.
+
+If the commit is done directly by using git (as opposed to
+using git-cvsserver) the update will need to happen on the
+next repository access by git-cvsserver, independent of
+access method and requested operation.
+
+That means that even if you offer only read access (e.g. by using
+the pserver method), git-cvsserver should have write access to
+the database to work reliably (otherwise you need to make sure
+that the database if up-to-date all the time git-cvsserver is run).
+
+By default it uses SQLite databases in the git directory, named
+`gitcvs.<module_name>.sqlite`. Note that the SQLite backend creates
+temporary files in the same directory as the database file on
+write so it might not be enough to grant the users using
+git-cvsserver write access to the database file without granting
+them write access to the directory, too.
+
+You can configure the database backend with the following
+configuration variables:
+
+Configuring database backend
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+git-cvsserver uses the Perl DBI module. Please also read
+its documentation if changing these variables, especially
+about `DBI->connect()`.
+
+gitcvs.dbname::
+       Database name. The exact meaning depends on the
+       used database driver, for SQLite this is a filename.
+       Supports variable substitution (see below). May
+       not contain semicolons (`;`).
+       Default: '%Ggitcvs.%m.sqlite'
+
+gitcvs.dbdriver::
+       Used DBI driver. You can specify any available driver
+       for this here, but it might not work. cvsserver is tested
+       with 'DBD::SQLite', reported to work with
+       'DBD::Pg', and reported *not* to work with 'DBD::mysql'.
+       Please regard this as an experimental feature. May not
+       contain double colons (`:`).
+       Default: 'SQLite'
+
+gitcvs.dbuser::
+       Database user. Only useful if setting `dbdriver`, since
+       SQLite has no concept of database users. Supports variable
+       substitution (see below).
+
+gitcvs.dbpass::
+       Database password.  Only useful if setting `dbdriver`, since
+       SQLite has no concept of database passwords.
+
+All variables can also be set per access method, see <<configaccessmethod,above>>.
+
+Variable substitution
+^^^^^^^^^^^^^^^^^^^^^
+In `dbdriver` and `dbuser` you can use the following variables:
+
+%G::
+       git directory name
+%g::
+       git directory name, where all characters except for
+       alpha-numeric ones, `.`, and `-` are replaced with
+       `_` (this should make it easier to use the directory
+       name in a filename if wanted)
+%m::
+       CVS module/git head name
+%a::
+       access method (one of "ext" or "pserver")
+%u::
+       Name of the user running git-cvsserver.
+       If no name can be determined, the
+       numeric uid is used.
+
 Eclipse CVS Client Notes
 ------------------------
 

diff --git a/git-cvsserver.perl b/git-cvsserver.perl
index 25816c5a21285cae1d8e42f46f26126a03c6631f..087e3abaefd54c8a950ce11e889757e48c428808 100755 (executable)
--- a/git-cvsserver.perl
+++ b/git-cvsserver.perl
@@ -91,7 +91,9 @@ $log->debug("Temporary directory is '$TEMP_DIR'");
 # if we are called with a pserver argument,
 # deal with the authentication cat before entering the
 # main loop
+$state->{method} = 'ext';
 if (@ARGV && $ARGV[0] eq 'pserver') {
+    $state->{method} = 'pserver';
     my $line = <STDIN>; chomp $line;
     unless( $line eq 'BEGIN AUTH REQUEST') {
        die "E Do not understand $line - expecting BEGIN AUTH REQUEST\n";
@@ -181,11 +183,18 @@ sub req_Root
     }
     foreach my $line ( @gitvars )
     {
-        next unless ( $line =~ /^(.*?)\.(.*?)=(.*)$/ );
-        $cfg->{$1}{$2} = $3;
+        next unless ( $line =~ /^(.*?)\.(.*?)(?:\.(.*?))?=(.*)$/ );
+        unless ($3) {
+            $cfg->{$1}{$2} = $4;
+        } else {
+            $cfg->{$1}{$2}{$3} = $4;
+        }
     }
 
-    unless ( defined ( $cfg->{gitcvs}{enabled} ) and $cfg->{gitcvs}{enabled} =~ /^\s*(1|true|yes)\s*$/i )
+    unless ( ($cfg->{gitcvs}{$state->{method}}{enabled}
+             and $cfg->{gitcvs}{$state->{method}}{enabled} =~ /^\s*(1|true|yes)\s*$/i)
+            or ($cfg->{gitcvs}{enabled}
+             and $cfg->{gitcvs}{enabled} =~ /^\s*(1|true|yes)\s*$/i) )
     {
         print "E GITCVS emulation needs to be enabled on this repo\n";
         print "E the repo config file needs a [gitcvs] section added, and the parameter 'enabled' set to 1\n";
@@ -194,9 +203,10 @@ sub req_Root
         return 0;
     }
 
-    if ( defined ( $cfg->{gitcvs}{logfile} ) )
+    my $logfile = $cfg->{gitcvs}{$state->{method}}{logfile} || $cfg->{gitcvs}{logfile};
+    if ( $logfile )
     {
-        $log->setfile($cfg->{gitcvs}{logfile});
+        $log->setfile($logfile);
     } else {
         $log->nofile();
     }
@@ -350,12 +360,52 @@ sub req_add
 
     argsplit("add");
 
+    my $updater = GITCVS::updater->new($state->{CVSROOT}, $state->{module}, $log);
+    $updater->update();
+
+    argsfromdir($updater);
+
     my $addcount = 0;
 
     foreach my $filename ( @{$state->{args}} )
     {
         $filename = filecleanup($filename);
 
+        my $meta = $updater->getmeta($filename);
+        my $wrev = revparse($filename);
+
+        if ($wrev && $meta && ($wrev < 0))
+        {
+            # previously removed file, add back
+            $log->info("added file $filename was previously removed, send 1.$meta->{revision}");
+
+            print "MT +updated\n";
+            print "MT text U \n";
+            print "MT fname $filename\n";
+            print "MT newline\n";
+            print "MT -updated\n";
+
+            unless ( $state->{globaloptions}{-n} )
+            {
+                my ( $filepart, $dirpart ) = filenamesplit($filename,1);
+
+                print "Created $dirpart\n";
+                print $state->{CVSROOT} . "/$state->{module}/$filename\n";
+
+                # this is an "entries" line
+                my $kopts = kopts_from_path($filepart);
+                $log->debug("/$filepart/1.$meta->{revision}//$kopts/");
+                print "/$filepart/1.$meta->{revision}//$kopts/\n";
+                # permissions
+                $log->debug("SEND : u=$meta->{mode},g=$meta->{mode},o=$meta->{mode}");
+                print "u=$meta->{mode},g=$meta->{mode},o=$meta->{mode}\n";
+                # transmit file
+                transmitfile($meta->{filehash});
+            }
+
+            next;
+        }
+
         unless ( defined ( $state->{entries}{$filename}{modified_filename} ) )
         {
             print "E cvs add: nothing known about `$filename'\n";
@@ -1027,7 +1077,7 @@ sub req_ci
 
     $log->info("req_ci : " . ( defined($data) ? $data : "[NULL]" ));
 
-    if ( @ARGV && $ARGV[0] eq 'pserver')
+    if ( $state->{method} eq 'pserver')
     {
         print "error 1 pserver access cannot commit\n";
         exit;
@@ -2132,25 +2182,40 @@ sub new
 
     bless $self, $class;
 
-    $self->{dbdir} = $config . "/";
-    die "Database dir '$self->{dbdir}' isn't a directory" unless ( defined($self->{dbdir}) and -d $self->{dbdir} );
-
     $self->{module} = $module;
-    $self->{file} = $self->{dbdir} . "/gitcvs.$module.sqlite";
-
     $self->{git_path} = $config . "/";
 
     $self->{log} = $log;
 
     die "Git repo '$self->{git_path}' doesn't exist" unless ( -d $self->{git_path} );
 
-    $self->{dbh} = DBI->connect("dbi:SQLite:dbname=" . $self->{file},"","");
+    $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} ||
+        $cfg->{gitcvs}{dbdriver} || "SQLite";
+    $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||
+        $cfg->{gitcvs}{dbname} || "%Ggitcvs.%m.sqlite";
+    $self->{dbuser} = $cfg->{gitcvs}{$state->{method}}{dbuser} ||
+        $cfg->{gitcvs}{dbuser} || "";
+    $self->{dbpass} = $cfg->{gitcvs}{$state->{method}}{dbpass} ||
+        $cfg->{gitcvs}{dbpass} || "";
+    my %mapping = ( m => $module,
+                    a => $state->{method},
+                    u => getlogin || getpwuid($<) || $<,
+                    G => $self->{git_path},
+                    g => mangle_dirname($self->{git_path}),
+                    );
+    $self->{dbname} =~ s/%([mauGg])/$mapping{$1}/eg;
+    $self->{dbuser} =~ s/%([mauGg])/$mapping{$1}/eg;
+
+    die "Invalid char ':' in dbdriver" if $self->{dbdriver} =~ /:/;
+    die "Invalid char ';' in dbname" if $self->{dbname} =~ /;/;
+    $self->{dbh} = DBI->connect("dbi:$self->{dbdriver}:dbname=$self->{dbname}",
+                                $self->{dbuser},
+                                $self->{dbpass});
+    die "Error connecting to database\n" unless defined $self->{dbh};
 
     $self->{tables} = {};
-    foreach my $table ( $self->{dbh}->tables )
+    foreach my $table ( keys %{$self->{dbh}->table_info(undef,undef,undef,'TABLE')->fetchall_hashref('TABLE_NAME')} )
     {
-        $table =~ s/^"//;
-        $table =~ s/"$//;
         $self->{tables}{$table} = 1;
     }
 
@@ -2848,5 +2913,19 @@ sub safe_pipe_capture {
     return wantarray ? @output : join('',@output);
 }
 
+=head2 mangle_dirname
+
+create a string from a directory name that is suitable to use as
+part of a filename, mainly by converting all chars except \w.- to _
+
+=cut
+sub mangle_dirname {
+    my $dirname = shift;
+    return unless defined $dirname;
+
+    $dirname =~ s/[^\w.-]/_/g;
+
+    return $dirname;
+}
 
 1;