![[tokkee]](http://tokkee.org/images/avatar.png){kind=avatar}
summary | shortlog | log | commit | commitdiff | tree
raw | patch | inline | side by side (parent: ee85cbc)
raw | patch | inline | side by side (parent: ee85cbc)
| author | Junio C Hamano <junkio@cox.net> | |
| Mon, 27 Jun 2005 10:33:33 +0000 (03:33 -0700) | ||
| committer | Linus Torvalds <torvalds@ppc970.osdl.org> | |
| Mon, 27 Jun 2005 22:27:51 +0000 (15:27 -0700) |
Packed delta files created by git-pack-objects seems to be the
way to go, and existing "delta" object handling code has exposed
the object representation details to too many places. Remove it
while we refactor code to come up with a proper interface in
sha1_file.c.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
way to go, and existing "delta" object handling code has exposed
the object representation details to too many places. Remove it
while we refactor code to come up with a proper interface in
sha1_file.c.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
25 files changed:
index ff97504cd6b0d6da8f686b4b5c22f96a7888e6c1..0ef01c3ffcf57d58715439785da7ecb743a31fa2 100644 (file)
SYNOPSIS
--------
-'git-fsck-cache' [--tags] [--root] [--delta-depth] [--unreachable] [--cache] [<object>*]
+'git-fsck-cache' [--tags] [--root] [--unreachable] [--cache] [<object>*]
DESCRIPTION
-----------
Consider any object recorded in the cache also as a head node for
an unreachability trace.
---delta-depth::
- Report back the length of the longest delta chain found.
-
It tests SHA1 and general object sanity, and it does full tracking of
the resulting reachability and everything else. It prints out any
corruption it finds (missing or bad objects), and if you use the
index 3ac91320e93bd773e2b212428968d83a0ee29ff3..431ef7cab3cadc94ca86e28c69611ee610a74767 100644 (file)
Get trees associated with the commit objects.
-a::
Get all the objects.
--d::
- Do not check for delta base objects (use this option
- only when you know the remote repository is not
- deltified).
---recover::
- Check dependency of deltified object more carefully than
- usual, to recover after earlier pull that was interrupted.
-v::
Report what is downloaded.
-w::
index 777bdbdd6e97a14a8e6c263b7e3c5271e53b8c04..cb9533401890012a36654eab7de99485798cb21c 100644 (file)
Get trees associated with the commit objects.
-a::
Get all the objects.
--d::
- Do not check for delta base objects (use this option
- only when you know the remote repository is not
- deltified).
---recover::
- Check dependency of deltified object more carefully than
- usual, to recover after earlier pull that was interrupted.
-v::
Report what is downloaded.
-w::
diff --git a/Documentation/git-mkdelta.txt b/Documentation/git-mkdelta.txt
+++ /dev/null
@@ -1,45 +0,0 @@
-git-mkdelta(1)
-==============
-May 2005
-
-NAME
-----
-git-mkdelta - Creates a delta object
-
-
-SYNOPSIS
---------
-'git-mkdelta' [-v] [-d N | --max-depth=N ] <reference_object> <target_object> [ <next_object> ... ]
-
-DESCRIPTION
------------
-Creates a delta object to replace <reference_object> by using an
-ordered list of potential objects to deltafy against earlier objects
-in the list.
-
-A cap on the depth of delta references can be provided as well,
-otherwise the default is to not have any limit. A limit of 0 will
-also undeltafy a given object.
-
-
-OPTIONS
--------
--v::
- Verbose
-
--d|--max-depth::
- limit the number of delta references in a chain
- If 0 then all objects are undeltafied.
-
-Author
-------
-Git is written by Linus Torvalds <torvalds@osdl.org> and the git-list <git@vger.kernel.org>.
-
-Documentation
---------------
-Documentation by David Greaves, Junio C Hamano and the git-list <git@vger.kernel.org>.
-
-GIT
----
-Part of the link:git.html[git] suite
-
index 3397fba8382955f232356efa5e8c6ae13e992433..ac3fb342f0522537850c03ead15939a0fd429465 100644 (file)
Get trees associated with the commit objects.
-a::
Get all the objects.
--d::
- Do not check for delta base objects (use this option
- only when you know the remote repository is not
- deltified).
---recover::
- Check dependency of deltified object more carefully than
- usual, to recover after earlier pull that was interrupted.
-v::
Report what is downloaded.
-w::
index 69013fd44a5b4cab09c93ebf530cbf05e3bc0574..e38679deb91d76f5ea327938205f67425de9e4b5 100644 (file)
Get tree associated with the requested commit object.
-a::
Get all the objects.
--d::
- Do not check for delta base objects (use this option
- only when you know the local repository is not
- deltified).
---recover::
- Check dependency of deltified object more carefully than
- usual, to recover after earlier push that was interrupted.
-v::
Report what is uploaded.
-w::
diff --git a/Documentation/git.txt b/Documentation/git.txt
index 971012bf830c23cd9267da881cc2d08ad4468260..5252ee8284a7fda8f653951261fe0367317b8643 100644 (file)
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
link:git-merge-base.html[git-merge-base]::
Finds as good a common ancestor as possible for a merge
-link:git-mkdelta.html[git-mkdelta]::
- Creates a delta object
-
link:git-mktag.html[git-mktag]::
Creates a tag object
diff --git a/Makefile b/Makefile
index e55d352146700eda4f36d30a777f4713eb3423aa..1736980d09a46e079eec4435d16d5b891865543e 100644 (file)
--- a/Makefile
+++ b/Makefile
SCRIPTS=git git-apply-patch-script git-merge-one-file-script git-prune-script \
git-pull-script git-tag-script git-resolve-script git-whatchanged \
- git-deltafy-script git-fetch-script git-status-script git-commit-script \
+ git-fetch-script git-status-script git-commit-script \
git-log-script git-shortlog git-cvsimport-script git-diff-script \
git-reset-script git-add-script git-checkout-script git-clone-script \
gitk git-cherry git-rebase-script git-relink-script
git-unpack-file git-export git-diff-cache git-convert-cache \
git-http-pull git-ssh-push git-ssh-pull git-rev-list git-mktag \
git-diff-helper git-tar-tree git-local-pull git-write-blob \
- git-get-tar-commit-id git-mkdelta git-apply git-stripspace \
+ git-get-tar-commit-id git-apply git-stripspace \
git-cvs2git git-diff-stages git-rev-parse git-patch-id \
git-pack-objects git-unpack-objects
$(INSTALL) $(PROG) $(SCRIPTS) $(dest)$(bin)
LIB_OBJS=read-cache.o sha1_file.o usage.o object.o commit.o tree.o blob.o \
- tag.o delta.o date.o index.o diff-delta.o patch-delta.o entry.o \
+ tag.o date.o index.o diff-delta.o patch-delta.o entry.o \
epoch.o refs.o csum-file.o
LIB_FILE=libgit.a
LIB_H=cache.h object.h blob.h tree.h commit.h tag.h delta.h epoch.h csum-file.h
git-diff-helper: diff-helper.c
git-tar-tree: tar-tree.c
git-write-blob: write-blob.c
-git-mkdelta: mkdelta.c
git-stripspace: stripspace.c
git-cvs2git: cvs2git.c
git-diff-stages: diff-stages.c
index f5deac7be59e7eeab8657fd9ae706fd6a57daed2..1b8d9b8b9771ce941a3a6f8f98fb8c197f61f4e0 100644 (file)
--- a/README
+++ b/README
All objects have a statically determined "type" aka "tag", which is
determined at object creation time, and which identifies the format of
the object (i.e. how it is used, and how it can refer to other
-objects). There are currently five different object types: "blob",
-"tree", "commit", "tag" and "delta"
+objects). There are currently four different object types: "blob",
+"tree", "commit" and "tag".
A "blob" object cannot refer to any other object, and is, like the tag
implies, a pure storage object containing some user data. It is used to
objects. It contains the identifier and type of another object, a
symbolic name (of course!) and, optionally, a signature.
-A "delta" object is used internally by the object database to minimise
-disk usage. Instead of storing the entire contents of a revision, git
-can behave in a similar manner to RCS et al and simply store a delta.
-
Regardless of object type, all objects share the following
characteristics: they are all deflated with zlib, and have a header
that not only specifies their tag, but also provides size information
about the data in the object. It's worth noting that the SHA1 hash
-that is used to name the object is the hash of the original data or
-the delta. (Historical note: in the dawn of the age of git the hash
+that is used to name the object is the hash of the original data.
+(Historical note: in the dawn of the age of git the hash
was the sha1 of the _compressed_ object)
As a result, the general consistency of an object can always be tested
A tag is created with link:git-mktag.html[git-mktag] and
it's data can be accessed by link:git-cat-file.html[git-cat-file]
-Delta Object
-~~~~~~~~~~~~
-
-The "delta" object is used internally by the object database to
-minimise storage usage by using xdeltas (byte level diffs). Deltas can
-form chains of arbitrary length as RCS does (although this is
-configureable at creation time). Most operations won't see or even be
-aware of delta objects as they are automatically 'applied' and appear
-as 'real' git objects In other words, if you write your own routines
-to look at the contents of the object database then you need to know
-about this - otherwise you don't. Actually, that's not quite true -
-one important area where deltas are likely to prove very valuable is
-in reducing bandwidth loads - so the more sophisticated network tools
-for git repositories will be aware of them too.
-
-Finally, git repositories can (and must) be deltafied in the
-background - the work to calculate the differences does not take place
-automatically at commit time.
-
-A delta can be created (or undeltafied) with
-link:git-mkdelta.html[git-mkdelta] it's raw data cannot be accessed at
-present.
-
The "index" aka "Current Directory Cache"
-----------------------------------------
index 2696e8e0e9a4c462ebca7efdc12f3cd0ddf7095f..765438d8f8c668f0294f084815d22105fab03a53 100644 (file)
--- a/cache.h
+++ b/cache.h
extern void * map_sha1_file(const unsigned char *sha1, unsigned long *size);
extern int unpack_sha1_header(z_stream *stream, void *map, unsigned long mapsize, void *buffer, unsigned long size);
extern int parse_sha1_header(char *hdr, char *type, unsigned long *sizep);
-extern int sha1_delta_base(const unsigned char *, unsigned char *);
extern int sha1_file_size(const unsigned char *, unsigned long *);
extern void * unpack_sha1_file(void *map, unsigned long mapsize, char *type, unsigned long *size);
extern void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size);
diff --git a/delta.c b/delta.c
--- a/delta.c
+++ /dev/null
@@ -1,114 +0,0 @@
-#include "object.h"
-#include "blob.h"
-#include "tree.h"
-#include "commit.h"
-#include "tag.h"
-#include "delta.h"
-#include "cache.h"
-
-/* the delta object definition (it can alias any other object) */
-struct delta {
- union {
- struct object object;
- struct blob blob;
- struct tree tree;
- struct commit commit;
- struct tag tag;
- } u;
-};
-
-struct delta *lookup_delta(const unsigned char *sha1)
-{
- struct object *obj = lookup_object(sha1);
- if (!obj) {
- struct delta *ret = xmalloc(sizeof(struct delta));
- memset(ret, 0, sizeof(struct delta));
- created_object(sha1, &ret->u.object);
- return ret;
- }
- return (struct delta *) obj;
-}
-
-int parse_delta_buffer(struct delta *item, void *buffer, unsigned long size)
-{
- struct object *reference;
- struct object_list *p;
-
- if (item->u.object.delta)
- return 0;
- item->u.object.delta = 1;
- if (size <= 20)
- return -1;
- reference = lookup_object(buffer);
- if (!reference) {
- struct delta *ref = xmalloc(sizeof(struct delta));
- memset(ref, 0, sizeof(struct delta));
- created_object(buffer, &ref->u.object);
- reference = &ref->u.object;
- }
-
- p = xmalloc(sizeof(*p));
- p->item = &item->u.object;
- p->next = reference->attached_deltas;
- reference->attached_deltas = p;
- return 0;
-}
-
-int process_deltas(void *src, unsigned long src_size, const char *src_type,
- struct object_list *delta_list)
-{
- int deepest = 0;
- do {
- struct object *obj = delta_list->item;
- static char type[10];
- void *map, *delta, *buf;
- unsigned long map_size, delta_size, buf_size;
- map = map_sha1_file(obj->sha1, &map_size);
- if (!map)
- continue;
- delta = unpack_sha1_file(map, map_size, type, &delta_size);
- munmap(map, map_size);
- if (!delta)
- continue;
- if (strcmp(type, "delta") || delta_size <= 20) {
- free(delta);
- continue;
- }
- buf = patch_delta(src, src_size,
- delta+20, delta_size-20,
- &buf_size);
- free(delta);
- if (!buf)
- continue;
- if (check_sha1_signature(obj->sha1, buf, buf_size, src_type) < 0)
- printf("sha1 mismatch for delta %s\n", sha1_to_hex(obj->sha1));
- if (obj->type && obj->type != src_type) {
- error("got %s when expecting %s for delta %s",
- src_type, obj->type, sha1_to_hex(obj->sha1));
- free(buf);
- continue;
- }
- obj->type = src_type;
- if (src_type == blob_type) {
- parse_blob_buffer((struct blob *)obj, buf, buf_size);
- } else if (src_type == tree_type) {
- parse_tree_buffer((struct tree *)obj, buf, buf_size);
- } else if (src_type == commit_type) {
- parse_commit_buffer((struct commit *)obj, buf, buf_size);
- } else if (src_type == tag_type) {
- parse_tag_buffer((struct tag *)obj, buf, buf_size);
- } else {
- error("unknown object type %s", src_type);
- free(buf);
- continue;
- }
- if (obj->attached_deltas) {
- int depth = process_deltas(buf, buf_size, src_type,
- obj->attached_deltas);
- if (deepest < depth)
- deepest = depth;
- }
- free(buf);
- } while ((delta_list = delta_list->next));
- return deepest + 1;
-}
index ccc0c9ecebfa160f1f33ce3285deaa3557814941..bc3a220179cefd0af169ab4f2e6aeeb5675f900a 100644 (file)
--- a/delta.h
+++ b/delta.h
void *delta_buf, unsigned long delta_size,
unsigned long *dst_size);
-/* handling of delta objects */
-struct delta;
-struct object_list;
-extern struct delta *lookup_delta(const unsigned char *sha1);
-extern int parse_delta_buffer(struct delta *item, void *buffer, unsigned long size);
-extern int parse_delta(struct delta *item, unsigned char sha1);
-extern int process_deltas(void *src, unsigned long src_size,
- const char *src_type, struct object_list *delta);
-
#endif
diff --git a/fsck-cache.c b/fsck-cache.c
index 7c55fe56a439d4ac02c288cb99c3d7b877751322..916792417d55b16124f2c352bfc1efcb8461ebc0 100644 (file)
--- a/fsck-cache.c
+++ b/fsck-cache.c
#include "tree.h"
#include "blob.h"
#include "tag.h"
-#include "delta.h"
#define REACHABLE 0x0001
static int show_root = 0;
static int show_tags = 0;
static int show_unreachable = 0;
-static int show_max_delta_depth = 0;
static int keep_cache_objects = 0;
static unsigned char head_sha1[20];
-static void expand_deltas(void)
-{
- int i, max_depth = 0;
-
- /*
- * To be as efficient as possible we look for delta heads and
- * recursively process them going backward, and parsing
- * resulting objects along the way. This allows for processing
- * each delta objects only once regardless of the delta depth.
- */
- for (i = 0; i < nr_objs; i++) {
- struct object *obj = objs[i];
- if (obj->parsed && !obj->delta && obj->attached_deltas) {
- int depth = 0;
- char type[10];
- unsigned long size;
- void *buf = read_sha1_file(obj->sha1, type, &size);
- if (!buf)
- continue;
- depth = process_deltas(buf, size, obj->type,
- obj->attached_deltas);
- if (max_depth < depth)
- max_depth = depth;
- }
- }
- if (show_max_delta_depth)
- printf("maximum delta depth = %d\n", max_depth);
-}
-
static void check_connectivity(void)
{
int i;
struct object_list *refs;
if (!obj->parsed) {
- if (obj->delta)
- printf("unresolved delta %s\n",
- sha1_to_hex(obj->sha1));
- else
- printf("missing %s %s\n",
- obj->type, sha1_to_hex(obj->sha1));
+ printf("missing %s %s\n",
+ obj->type, sha1_to_hex(obj->sha1));
continue;
}
}
if (show_unreachable && !(obj->flags & REACHABLE)) {
- if (obj->attached_deltas)
- printf("foreign delta reference %s\n",
- sha1_to_hex(obj->sha1));
- else
- printf("unreachable %s %s\n",
- obj->type, sha1_to_hex(obj->sha1));
+ printf("unreachable %s %s\n",
+ obj->type, sha1_to_hex(obj->sha1));
continue;
}
return fsck_commit((struct commit *) obj);
if (obj->type == tag_type)
return fsck_tag((struct tag *) obj);
- if (!obj->type && obj->delta)
- return 0;
return -1;
}
show_root = 1;
continue;
}
- if (!strcmp(arg, "--delta-depth")) {
- show_max_delta_depth = 1;
- continue;
- }
if (!strcmp(arg, "--cache")) {
keep_cache_objects = 1;
continue;
}
fsck_sha1_list();
- expand_deltas();
-
heads = 0;
for (i = 1; i < argc; i++) {
const char *arg = argv[i];
diff --git a/git-deltafy-script b/git-deltafy-script
--- a/git-deltafy-script
+++ /dev/null
@@ -1,81 +0,0 @@
-#!/bin/bash
-
-# Example script to deltify an entire GIT repository based on the commit list.
-# The most recent version of a file is the reference and previous versions
-# are made delta against the best earlier version available. And so on for
-# successive versions going back in time. This way the increasing delta
-# overhead is pushed towards older versions of any given file.
-#
-# The -d argument allows to provide a limit on the delta chain depth.
-# If 0 is passed then everything is undeltafied. Limiting the delta
-# depth is meaningful for subsequent access performance to old revisions.
-# A value of 16 might be a good compromize between performance and good
-# space saving. Current default is unbounded.
-#
-# The --max-behind=30 argument is passed to git-mkdelta so to keep
-# combinations and memory usage bounded a bit. If you have lots of memory
-# and CPU power you may remove it (or set to 0) to let git-mkdelta find the
-# best delta match regardless of the number of revisions for a given file.
-# You can also make the value smaller to make it faster and less
-# memory hungry. A value of 5 ought to still give pretty good results.
-# When set to 0 or ommitted then look behind is unbounded. Note that
-# git-mkdelta might die with a segmentation fault in that case if it
-# runs out of memory. Note that the GIT repository will still be consistent
-# even if git-mkdelta dies unexpectedly.
-
-set -e
-
-max_depth=
-[ "$1" == "-d" ] && max_depth="--max-depth=$2" && shift 2
-
-overlap=30
-max_behind="--max-behind=$overlap"
-
-function process_list() {
- if [ "$list" ]; then
- echo "Processing $curr_file"
- echo "$list" | xargs git-mkdelta $max_depth $max_behind -v
- fi
-}
-
-rev_list=""
-curr_file=""
-
-git-rev-list HEAD |
-while true; do
- # Let's batch revisions into groups of 1000 to give it a chance to
- # scale with repositories containing long revision lists. We also
- # overlap with the previous batch the size of mkdelta's look behind
- # value in order to account for the processing discontinuity.
- rev_list="$(echo -e -n "$rev_list" | tail --lines=$overlap)"
- for i in $(seq 1000); do
- read rev || break
- rev_list="$rev_list$rev\n"
- done
- echo -e -n "$rev_list" |
- git-diff-tree -r -t --stdin |
- awk '/^:/ { if ($5 == "M") printf "%s %s\n%s %s\n", $4, $6, $3, $6 }' |
- LC_ALL=C sort -s -k 2 | uniq |
- while read sha1 file; do
- if [ "$file" == "$curr_file" ]; then
- list="$list $sha1"
- else
- process_list
- curr_file="$file"
- list="$sha1"
- fi
- done
- [ "$rev" ] || break
-done
-process_list
-
-curr_file="root directory"
-list="$(
- git-rev-list HEAD |
- while read commit; do
- git-cat-file commit $commit |
- sed -n 's/tree //p;Q'
- done
- )"
-process_list
-
diff --git a/http-pull.c b/http-pull.c
index ec53dad8efbe6e7734b75eaba8821ac290a5abbb..1f9d60b9b1d5eed85b24d96c240666bbfc5a22ed 100644 (file)
--- a/http-pull.c
+++ b/http-pull.c
get_tree = 1;
} else if (argv[arg][1] == 'c') {
get_history = 1;
- } else if (argv[arg][1] == 'd') {
- get_delta = 0;
- } else if (!strcmp(argv[arg], "--recover")) {
- get_delta = 2;
} else if (argv[arg][1] == 'a') {
get_all = 1;
get_tree = 1;
diff --git a/local-pull.c b/local-pull.c
index 535bd8c297b51edd47b0477affcadf8b21e7f6bf..2f06fbee8b840a7ae642f5a22e2cb993687f3470 100644 (file)
--- a/local-pull.c
+++ b/local-pull.c
get_tree = 1;
else if (argv[arg][1] == 'c')
get_history = 1;
- else if (argv[arg][1] == 'd')
- get_delta = 0;
- else if (!strcmp(argv[arg], "--recover"))
- get_delta = 2;
else if (argv[arg][1] == 'a') {
get_all = 1;
get_tree = 1;
diff --git a/mkdelta.c b/mkdelta.c
--- a/mkdelta.c
+++ /dev/null
@@ -1,364 +0,0 @@
-/*
- * Deltafication of a GIT database.
- *
- * (C) 2005 Nicolas Pitre <nico@cam.org>
- *
- * This code is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#include "cache.h"
-#include "delta.h"
-
-static int replace_object(char *buf, unsigned long size, unsigned char *sha1)
-{
- char tmpfile[PATH_MAX];
- int fd;
-
- snprintf(tmpfile, sizeof(tmpfile), "%s/obj_XXXXXX", get_object_directory());
- fd = mkstemp(tmpfile);
- if (fd < 0)
- return error("%s: %s\n", tmpfile, strerror(errno));
- if (write(fd, buf, size) != size) {
- perror("unable to write file");
- close(fd);
- unlink(tmpfile);
- return -1;
- }
- fchmod(fd, 0444);
- close(fd);
- if (rename(tmpfile, sha1_file_name(sha1))) {
- perror("unable to replace original object");
- unlink(tmpfile);
- return -1;
- }
- return 0;
-}
-
-static void *create_object(unsigned char *buf, unsigned long len,
- char *hdr, int hdrlen, unsigned long *retsize)
-{
- unsigned char *compressed;
- unsigned long size;
- z_stream stream;
-
- /* Set it up */
- memset(&stream, 0, sizeof(stream));
- deflateInit(&stream, Z_BEST_COMPRESSION);
- size = deflateBound(&stream, len+hdrlen);
- compressed = xmalloc(size);
-
- /* Compress it */
- stream.next_out = compressed;
- stream.avail_out = size;
-
- /* First header.. */
- stream.next_in = (unsigned char *)hdr;
- stream.avail_in = hdrlen;
- while (deflate(&stream, 0) == Z_OK)
- /* nothing */;
-
- /* Then the data itself.. */
- stream.next_in = buf;
- stream.avail_in = len;
- while (deflate(&stream, Z_FINISH) == Z_OK)
- /* nothing */;
- deflateEnd(&stream);
- *retsize = stream.total_out;
- return compressed;
-}
-
-static int restore_original_object(unsigned char *buf, unsigned long len,
- char *type, unsigned char *sha1)
-{
- char hdr[50];
- int hdrlen, ret;
- void *compressed;
- unsigned long size;
-
- hdrlen = sprintf(hdr, "%s %lu", type, len)+1;
- compressed = create_object(buf, len, hdr, hdrlen, &size);
- ret = replace_object(compressed, size, sha1);
- free(compressed);
- return ret;
-}
-
-static void *create_delta_object(unsigned char *buf, unsigned long len,
- unsigned char *sha1_ref, unsigned long *size)
-{
- char hdr[50];
- int hdrlen;
-
- /* Generate the header + sha1 of reference for delta */
- hdrlen = sprintf(hdr, "delta %lu", len+20)+1;
- memcpy(hdr + hdrlen, sha1_ref, 20);
- hdrlen += 20;
-
- return create_object(buf, len, hdr, hdrlen, size);
-}
-
-static void *get_buffer(unsigned char *sha1, char *type,
- unsigned long *size, unsigned long *compsize)
-{
- unsigned long mapsize;
- void *map = map_sha1_file(sha1, &mapsize);
- if (map) {
- void *buffer = unpack_sha1_file(map, mapsize, type, size);
- munmap(map, mapsize);
- if (compsize)
- *compsize = mapsize;
- if (buffer)
- return buffer;
- }
- error("unable to get object %s", sha1_to_hex(sha1));
- return NULL;
-}
-
-static void *expand_delta(void *delta, unsigned long *size, char *type,
- unsigned int *depth, unsigned char **links)
-{
- void *buf = NULL;
- unsigned int level = (*depth)++;
- if (*size < 20) {
- error("delta object is bad");
- free(delta);
- } else {
- unsigned long ref_size;
- void *ref = get_buffer(delta, type, &ref_size, NULL);
- if (ref && !strcmp(type, "delta"))
- ref = expand_delta(ref, &ref_size, type, depth, links);
- else if (ref)
-{
- *links = xmalloc(*depth * 20);
-}
- if (ref) {
- buf = patch_delta(ref, ref_size, delta+20, *size-20, size);
- free(ref);
- if (buf)
- memcpy(*links + level*20, delta, 20);
- else
- free(*links);
- }
- free(delta);
- }
- return buf;
-}
-
-static char *mkdelta_usage =
-"mkdelta [--max-depth=N] [--max-behind=N] <reference_sha1> <target_sha1> [<next_sha1> ...]";
-
-struct delta {
- unsigned char sha1[20]; /* object sha1 */
- unsigned long size; /* object size */
- void *buf; /* object content */
- unsigned char *links; /* delta reference links */
- unsigned int depth; /* delta depth */
-};
-
-int main(int argc, char **argv)
-{
- struct delta *ref, trg;
- char ref_type[20], trg_type[20], *skip_reason;
- void *best_buf;
- unsigned long best_size, orig_size, orig_compsize;
- unsigned int r, orig_ref, best_ref, nb_refs, next_ref, max_refs = 0;
- unsigned int i, duplicate, skip_lvl, verbose = 0, quiet = 0;
- unsigned int max_depth = -1;
-
- for (i = 1; i < argc; i++) {
- if (!strcmp(argv[i], "-v")) {
- verbose = 1;
- quiet = 0;
- } else if (!strcmp(argv[i], "-q")) {
- quiet = 1;
- verbose = 0;
- } else if (!strcmp(argv[i], "-d") && i+1 < argc) {
- max_depth = atoi(argv[++i]);
- } else if (!strncmp(argv[i], "--max-depth=", 12)) {
- max_depth = atoi(argv[i]+12);
- } else if (!strcmp(argv[i], "-b") && i+1 < argc) {
- max_refs = atoi(argv[++i]);
- } else if (!strncmp(argv[i], "--max-behind=", 13)) {
- max_refs = atoi(argv[i]+13);
- } else
- break;
- }
-
- if (i + (max_depth != 0) >= argc)
- usage(mkdelta_usage);
-
- if (!max_refs || max_refs > argc - i)
- max_refs = argc - i;
- ref = xmalloc(max_refs * sizeof(*ref));
- for (r = 0; r < max_refs; r++)
- ref[r].buf = ref[r].links = NULL;
- next_ref = nb_refs = 0;
-
- do {
- if (get_sha1(argv[i], trg.sha1))
- die("bad sha1 %s", argv[i]);
- trg.buf = get_buffer(trg.sha1, trg_type, &trg.size, &orig_compsize);
- if (trg.buf && !trg.size) {
- if (verbose)
- printf("skip %s (object is empty)\n", argv[i]);
- continue;
- }
- orig_size = trg.size;
- orig_ref = -1;
- trg.depth = 0;
- trg.links = NULL;
- if (trg.buf && !strcmp(trg_type, "delta")) {
- for (r = 0; r < nb_refs; r++)
- if (!memcmp(trg.buf, ref[r].sha1, 20))
- break;
- if (r < nb_refs) {
- /* no need to reload the reference object */
- trg.depth = ref[r].depth + 1;
- trg.links = xmalloc(trg.depth*20);
- memcpy(trg.links, trg.buf, 20);
- memcpy(trg.links+20, ref[r].links, ref[r].depth*20);
- trg.buf = patch_delta(ref[r].buf, ref[r].size,
- trg.buf+20, trg.size-20,
- &trg.size);
- strcpy(trg_type, ref_type);
- orig_ref = r;
- } else {
- trg.buf = expand_delta(trg.buf, &trg.size, trg_type,
- &trg.depth, &trg.links);
- }
- }
- if (!trg.buf)
- die("unable to read target object %s", argv[i]);
-
- if (!nb_refs) {
- strcpy(ref_type, trg_type);
- } else if (max_depth && strcmp(ref_type, trg_type)) {
- die("type mismatch for object %s", argv[i]);
- }
-
- duplicate = 0;
- best_buf = NULL;
- best_size = -1;
- best_ref = -1;
- skip_lvl = 0;
- skip_reason = NULL;
- for (r = 0; max_depth && r < nb_refs; r++) {
- void *delta_buf, *comp_buf;
- unsigned long delta_size, comp_size;
- unsigned int l;
-
- duplicate = !memcmp(trg.sha1, ref[r].sha1, 20);
- if (duplicate) {
- skip_reason = "already seen";
- break;
- }
- if (ref[r].depth >= max_depth) {
- if (skip_lvl < 1) {
- skip_reason = "exceeding max link depth";
- skip_lvl = 1;
- }
- continue;
- }
- for (l = 0; l < ref[r].depth; l++)
- if (!memcmp(trg.sha1, ref[r].links + l*20, 20))
- break;
- if (l != ref[r].depth) {
- if (skip_lvl < 2) {
- skip_reason = "would create a loop";
- skip_lvl = 2;
- }
- continue;
- }
- if (trg.depth < max_depth && r == orig_ref) {
- if (skip_lvl < 3) {
- skip_reason = "delta already in place";
- skip_lvl = 3;
- }
- continue;
- }
- delta_buf = diff_delta(ref[r].buf, ref[r].size,
- trg.buf, trg.size,
- &delta_size, ~0UL);
- if (!delta_buf)
- die("out of memory");
- if (trg.depth < max_depth &&
- delta_size+20 >= orig_size) {
- /* no need to even try to compress if original
- object is smaller than this delta */
- free(delta_buf);
- if (skip_lvl < 4) {
- skip_reason = "no size reduction";
- skip_lvl = 4;
- }
- continue;
- }
- comp_buf = create_delta_object(delta_buf, delta_size,
- ref[r].sha1, &comp_size);
- if (!comp_buf)
- die("out of memory");
- free(delta_buf);
- if (trg.depth < max_depth &&
- comp_size >= orig_compsize) {
- free(comp_buf);
- if (skip_lvl < 5) {
- skip_reason = "no size reduction";
- skip_lvl = 5;
- }
- continue;
- }
- if ((comp_size < best_size) ||
- (comp_size == best_size &&
- ref[r].depth < ref[best_ref].depth)) {
- free(best_buf);
- best_buf = comp_buf;
- best_size = comp_size;
- best_ref = r;
- }
- }
-
- if (best_buf) {
- if (replace_object(best_buf, best_size, trg.sha1))
- die("unable to write delta for %s", argv[i]);
- free(best_buf);
- free(trg.links);
- trg.depth = ref[best_ref].depth + 1;
- trg.links = xmalloc(trg.depth*20);
- memcpy(trg.links, ref[best_ref].sha1, 20);
- memcpy(trg.links+20, ref[best_ref].links, ref[best_ref].depth*20);
- if (!quiet)
- printf("delta %s (size=%ld.%02ld%% depth=%d dist=%d)\n",
- argv[i], best_size*100 / orig_compsize,
- (best_size*10000 / orig_compsize)%100,
- trg.depth,
- (next_ref - best_ref + max_refs)
- % (max_refs + 1) + 1);
- } else if (trg.depth > max_depth) {
- if (restore_original_object(trg.buf, trg.size, trg_type, trg.sha1))
- die("unable to restore %s", argv[i]);
- if (!quiet)
- printf("undelta %s (depth was %d)\n",
- argv[i], trg.depth);
- trg.depth = 0;
- free(trg.links);
- trg.links = NULL;
- } else if (skip_reason && verbose) {
- printf("skip %s (%s)\n", argv[i], skip_reason);
- }
-
- if (!duplicate) {
- free(ref[next_ref].buf);
- free(ref[next_ref].links);
- ref[next_ref] = trg;
- if (++next_ref > nb_refs)
- nb_refs = next_ref;
- if (next_ref == max_refs)
- next_ref = 0;
- } else {
- free(trg.buf);
- free(trg.links);
- }
- } while (++i < argc);
-
- return 0;
-}
diff --git a/object.c b/object.c
index 21f872ee163e9eeff1a52854791c5b96a2ec2ceb..c410e97c8f378bd6c680efa48c82c3d3f7907103 100644 (file)
--- a/object.c
+++ b/object.c
#include "commit.h"
#include "cache.h"
#include "tag.h"
-#include "delta.h"
struct object **objs;
int nr_objs;
@@ -116,25 +115,14 @@ struct object *lookup_object_type(const unsigned char *sha1, const char *type)
struct object *parse_object(const unsigned char *sha1)
{
- unsigned long mapsize;
- void *map = map_sha1_file(sha1, &mapsize);
- if (map) {
- int is_delta;
+ unsigned long size;
+ char type[20];
+ void *buffer = read_sha1_file(sha1, type, &size);
+ if (buffer) {
struct object *obj;
- char type[100];
- unsigned long size;
- void *buffer = unpack_sha1_file(map, mapsize, type, &size);
- munmap(map, mapsize);
- if (!buffer)
- return NULL;
- is_delta = !strcmp(type, "delta");
- if (!is_delta && check_sha1_signature(sha1, buffer, size, type) < 0)
+ if (check_sha1_signature(sha1, buffer, size, type) < 0)
printf("sha1 mismatch %s\n", sha1_to_hex(sha1));
- if (is_delta) {
- struct delta *delta = lookup_delta(sha1);
- parse_delta_buffer(delta, buffer, size);
- obj = (struct object *) delta;
- } else if (!strcmp(type, "blob")) {
+ if (!strcmp(type, "blob")) {
struct blob *blob = lookup_blob(sha1);
parse_blob_buffer(blob, buffer, size);
obj = &blob->object;
diff --git a/object.h b/object.h
index 9ad4ae83e12ab84c94eacfa3a8db1bd01e987c2a..d82c7c454bfc64e75b74f03c2d34d3f7e778a0f1 100644 (file)
--- a/object.h
+++ b/object.h
struct object {
unsigned parsed : 1;
unsigned used : 1;
- unsigned delta : 1;
unsigned int flags;
unsigned char sha1[20];
const char *type;
struct object_list *refs;
- struct object_list *attached_deltas;
void *util;
};
diff --git a/pack-objects.c b/pack-objects.c
index 3fe3b05d51e7960d978a9efb765cfcd8d8a43009..102af3054bc27e864117663628c69a5beb8c26a5 100644 (file)
--- a/pack-objects.c
+++ b/pack-objects.c
OBJ_COMMIT,
OBJ_TREE,
OBJ_BLOB,
- OBJ_DELTA // NOTE! This is _not_ the same as a "delta" object in the filesystem
+ OBJ_DELTA
};
struct object_entry {
index 91d9db6c7b1be84e7a5fe21c5194fbf22dadc8cb..ed3078e3b27c62c07558fd94f339801cbd685593 100644 (file)
--- a/pull.c
+++ b/pull.c
int get_tree = 0;
int get_history = 0;
-/* 1 means "get delta", 2 means "really check delta harder */
-int get_delta = 1;
int get_all = 0;
int get_verbosely = 0;
static unsigned char current_commit_sha1[20];
if (status && what)
report_missing(what, sha1);
}
- else if (get_delta < 2)
- return 0;
-
- if (get_delta) {
- unsigned char delta_sha1[20];
- status = sha1_delta_base(sha1, delta_sha1);
- if (0 < status)
- status = make_sure_we_have_it(what, delta_sha1);
- }
return status;
}
index ddcca60e21b836973bf8ce2a57f7b516bceae704..e173ae3337c4465da87d849f4e5c9da203fdf01d 100644 (file)
--- a/pull.h
+++ b/pull.h
/* Set to fetch the trees in the commit history. */
extern int get_all;
-/* Set to zero to skip the check for delta object base;
- * set to two to check delta dependency even for objects we already have.
- */
-extern int get_delta;
-
/* Set to be verbose */
extern int get_verbosely;
diff --git a/sha1_file.c b/sha1_file.c
index 6d6073da85d297b4b0a59c0548720b21fc23129d..a12da4e238cad6981507a96576bcf5cfad359f6b 100644 (file)
--- a/sha1_file.c
+++ b/sha1_file.c
* creation etc.
*/
#include "cache.h"
-#include "delta.h"
#ifndef O_NOATIME
#if defined(__linux__) && (defined(__i386__) || defined(__PPC__))
@@ -410,37 +409,6 @@ void * unpack_sha1_file(void *map, unsigned long mapsize, char *type, unsigned l
return unpack_sha1_rest(&stream, hdr, *size);
}
-int sha1_delta_base(const unsigned char *sha1, unsigned char *base_sha1)
-{
- int ret;
- unsigned long mapsize, size;
- void *map;
- z_stream stream;
- char hdr[64], type[20];
- void *delta_data_head;
-
- map = map_sha1_file(sha1, &mapsize);
- if (!map)
- return -1;
- ret = unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr));
- if (ret < Z_OK || parse_sha1_header(hdr, type, &size) < 0) {
- ret = -1;
- goto out;
- }
- if (strcmp(type, "delta")) {
- ret = 0;
- goto out;
- }
-
- delta_data_head = hdr + strlen(hdr) + 1;
- ret = 1;
- memcpy(base_sha1, delta_data_head, 20);
- out:
- inflateEnd(&stream);
- munmap(map, mapsize);
- return ret;
-}
-
int sha1_file_size(const unsigned char *sha1, unsigned long *sizep)
{
int ret, status;
void *map;
z_stream stream;
char hdr[64], type[20];
- const unsigned char *data;
- unsigned char cmd;
- int i;
map = map_sha1_file(sha1, &mapsize);
if (!map)
return -1;
ret = unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr));
- status = -1;
if (ret < Z_OK || parse_sha1_header(hdr, type, &size) < 0)
- goto out;
- if (strcmp(type, "delta")) {
- *sizep = size;
+ status = -1;
+ else {
status = 0;
- goto out;
- }
-
- /* We are dealing with a delta object. Inflated, the first
- * 20 bytes hold the base object SHA1, and delta data follows
- * immediately after it.
- *
- * The initial part of the delta starts at delta_data_head +
- * 20. Borrow code from patch-delta to read the result size.
- */
- data = (unsigned char *)(hdr + strlen(hdr) + 1 + 20);
-
- /* Skip over the source size; we are not interested in
- * it and we cannot verify it because we do not want
- * to read the base object.
- */
- cmd = *data++;
- while (cmd) {
- if (cmd & 1)
- data++;
- cmd >>= 1;
- }
- /* Read the result size */
- size = i = 0;
- cmd = *data++;
- while (cmd) {
- if (cmd & 1)
- size |= *data++ << i;
- i += 8;
- cmd >>= 1;
+ *sizep = size;
}
- *sizep = size;
- status = 0;
- out:
inflateEnd(&stream);
munmap(map, mapsize);
return status;
@@ -510,19 +441,6 @@ void * read_sha1_file(const unsigned char *sha1, char *type, unsigned long *size
if (map) {
buf = unpack_sha1_file(map, mapsize, type, size);
munmap(map, mapsize);
- if (buf && !strcmp(type, "delta")) {
- void *ref = NULL, *delta = buf;
- unsigned long ref_size, delta_size = *size;
- buf = NULL;
- if (delta_size > 20)
- ref = read_sha1_file(delta, type, &ref_size);
- if (ref)
- buf = patch_delta(ref, ref_size,
- delta+20, delta_size-20,
- size);
- free(delta);
- free(ref);
- }
return buf;
}
return NULL;
diff --git a/ssh-pull.c b/ssh-pull.c
index 27484126dd899cd9a6d4a14d2f41a60fc3a36a52..87d523899a83d8c0d3c5ff721208ded30c1a38f4 100644 (file)
--- a/ssh-pull.c
+++ b/ssh-pull.c
get_tree = 1;
} else if (argv[arg][1] == 'c') {
get_history = 1;
- } else if (argv[arg][1] == 'd') {
- get_delta = 0;
- } else if (!strcmp(argv[arg], "--recover")) {
- get_delta = 2;
} else if (argv[arg][1] == 'a') {
get_all = 1;
get_tree = 1;
diff --git a/t/t5100-delta-pull.sh b/t/t5100-delta-pull.sh
--- a/t/t5100-delta-pull.sh
+++ /dev/null
@@ -1,79 +0,0 @@
-#!/bin/sh
-#
-# Copyright (c) 2005 Junio C Hamano
-#
-
-test_description='Test pulling deltified objects
-
-'
-. ./test-lib.sh
-
-locate_obj='s|\(..\)|.git/objects/\1/|'
-
-test_expect_success \
- setup \
- 'cat ../README >a &&
- git-update-cache --add a &&
- a0=`git-ls-files --stage |
- sed -e '\''s/^[0-7]* \([0-9a-f]*\) .*/\1/'\''` &&
-
- sed -e 's/test/TEST/g' ../README >a &&
- git-update-cache a &&
- a1=`git-ls-files --stage |
- sed -e '\''s/^[0-7]* \([0-9a-f]*\) .*/\1/'\''` &&
- tree=`git-write-tree` &&
- commit=`git-commit-tree $tree </dev/null` &&
- a0f=`echo "$a0" | sed -e "$locate_obj"` &&
- a1f=`echo "$a1" | sed -e "$locate_obj"` &&
- echo commit $commit &&
- echo a0 $a0 &&
- echo a1 $a1 &&
- ls -l $a0f $a1f &&
- echo $commit >.git/HEAD &&
- git-mkdelta -v $a0 $a1 &&
- ls -l $a0f $a1f'
-
-# Now commit has a tree that records delitified "a" whose SHA1 is a1.
-# Create a new repo and pull this commit into it.
-
-test_expect_success \
- 'setup and cd into new repo' \
- 'mkdir dest && cd dest && rm -fr .git && git-init-db'
-
-test_expect_success \
- 'pull from deltified repo into a new repo without -d' \
- 'rm -fr .git a && git-init-db &&
- git-local-pull -v -a $commit ../.git/ &&
- git-cat-file blob $a1 >a &&
- diff -u a ../a'
-
-test_expect_failure \
- 'pull from deltified repo into a new repo with -d' \
- 'rm -fr .git a && git-init-db &&
- git-local-pull -v -a -d $commit ../.git/ &&
- git-cat-file blob $a1 >a &&
- diff -u a ../a'
-
-test_expect_failure \
- 'pull from deltified repo after delta failure without --recover' \
- 'rm -f a &&
- git-local-pull -v -a $commit ../.git/ &&
- git-cat-file blob $a1 >a &&
- diff -u a ../a'
-
-test_expect_success \
- 'pull from deltified repo after delta failure with --recover' \
- 'rm -f a &&
- git-local-pull -v -a --recover $commit ../.git/ &&
- git-cat-file blob $a1 >a &&
- diff -u a ../a'
-
-test_expect_success \
- 'missing-tree or missing-blob should be re-fetched without --recover' \
- 'rm -f a $a0f $a1f &&
- git-local-pull -v -a $commit ../.git/ &&
- git-cat-file blob $a1 >a &&
- diff -u a ../a'
-
-test_done
-