![[tokkee]](http://tokkee.org/images/avatar.png){kind=avatar}
| author | Junio C Hamano <gitster@pobox.com> | |
| Mon, 1 Feb 2010 20:41:31 +0000 (12:41 -0800) | ||
| committer | Junio C Hamano <gitster@pobox.com> | |
| Mon, 1 Feb 2010 20:42:00 +0000 (12:42 -0800) |
* sp/maint-fast-import-large-blob:
fast-import: Stream very large blobs directly to pack
bash: don't offer remote transport helpers as subcommands
Conflicts:
fast-import.c
fast-import: Stream very large blobs directly to pack
bash: don't offer remote transport helpers as subcommands
Conflicts:
fast-import.c
| 1 | 2 | |||
|---|---|---|---|---|
| Documentation/config.txt | patch | | diff1 | | diff2 | | blob | history |
| Documentation/git-fast-import.txt | patch | | diff1 | | diff2 | | blob | history |
| contrib/completion/git-completion.bash | patch | | diff1 | | diff2 | | blob | history |
| fast-import.c | patch | | diff1 | | diff2 | | blob | history |
| t/t9300-fast-import.sh | patch | | diff1 | | diff2 | | blob | history |
diff --combined Documentation/config.txt
index 17901e244a832efde1f6264957c8f9e8c2bd448a,b16a20bc3606153ab43a351c5a10e9ad2ecdb708..36ad992a56737676959c22b8df5b25d4b62ed989
+++ b/Documentation/config.txt
a string, an integer, or a boolean. Boolean values may be given as yes/no,
0/1, true/false or on/off. Case is not significant in boolean values, when
converting value to the canonical form using '--bool' type specifier;
-'git-config' will ensure that the output is "true" or "false".
+'git config' will ensure that the output is "true" or "false".
String values may be entirely or partially enclosed in double quotes.
You need to enclose variable values in double quotes if you want to
Advice shown when linkgit:git-merge[1] refuses to
merge to avoid overwritting local changes.
Default: true.
+ resolveConflict::
+ Advices shown by various commands when conflicts
+ prevent the operation from being performed.
+ Default: true.
+ implicitIdentity::
+ Advice on how to set your identity configuration when
+ your information is guessed from the system username and
+ domain name. Default: true.
--
core.fileMode::
+
Common unit suffixes of 'k', 'm', or 'g' are supported.
+ core.bigFileThreshold::
+ Files larger than this size are stored deflated, without
+ attempting delta compression. Storing large files without
+ delta compression avoids excessive memory usage, at the
+ slight expense of increased disk usage.
+ +
+ Default is 512 MiB on all platforms. This should be reasonable
+ for most projects as source code and other text files can still
+ be delta compressed, but larger binary media files won't be.
+ +
+ Common unit suffixes of 'k', 'm', or 'g' are supported.
+ +
+ Currently only linkgit:git-fast-import[1] honors this setting.
+
core.excludesfile::
In addition to '.gitignore' (per-directory) and
'.git/info/exclude', git looks into this file for patterns
core.whitespace::
A comma separated list of common whitespace problems to
- notice. 'git-diff' will use `color.diff.whitespace` to
- highlight them, and 'git-apply --whitespace=error' will
+ notice. 'git diff' will use `color.diff.whitespace` to
+ highlight them, and 'git apply --whitespace=error' will
consider them as errors. You can prefix `-` to disable
any of them (e.g. `-trailing-space`):
+
This setting defaults to "refs/notes/commits", and can be overridden by
the `GIT_NOTES_REF` environment variable.
+core.sparseCheckout::
+ Enable "sparse checkout" feature. See section "Sparse checkout" in
+ linkgit:git-read-tree[1] for more information.
+
add.ignore-errors::
- Tells 'git-add' to continue adding files when some files cannot be
+ Tells 'git add' to continue adding files when some files cannot be
added due to indexing errors. Equivalent to the '--ignore-errors'
option of linkgit:git-add[1].
not necessarily be the current directory.
apply.ignorewhitespace::
- When set to 'change', tells 'git-apply' to ignore changes in
+ When set to 'change', tells 'git apply' to ignore changes in
whitespace, in the same way as the '--ignore-space-change'
option.
- When set to one of: no, none, never, false tells 'git-apply' to
+ When set to one of: no, none, never, false tells 'git apply' to
respect all whitespace differences.
See linkgit:git-apply[1].
apply.whitespace::
- Tells 'git-apply' how to handle whitespaces, in the same way
+ Tells 'git apply' how to handle whitespaces, in the same way
as the '--whitespace' option. See linkgit:git-apply[1].
branch.autosetupmerge::
- Tells 'git-branch' and 'git-checkout' to set up new branches
+ Tells 'git branch' and 'git checkout' to set up new branches
so that linkgit:git-pull[1] will appropriately merge from the
starting point branch. Note that even if this option is not set,
this behavior can be chosen per-branch using the `--track`
branch. This option defaults to true.
branch.autosetuprebase::
- When a new branch is created with 'git-branch' or 'git-checkout'
+ When a new branch is created with 'git branch' or 'git checkout'
that tracks another branch, this variable tells git to set
up pull to rebase instead of merge (see "branch.<name>.rebase").
When `never`, rebase is never automatically set to true.
This option defaults to never.
branch.<name>.remote::
- When in branch <name>, it tells 'git-fetch' and 'git-push' which
+ When in branch <name>, it tells 'git fetch' and 'git push' which
remote to fetch from/push to. It defaults to `origin` if no remote is
configured. `origin` is also used if you are not on any branch.
branch.<name>.merge::
Defines, together with branch.<name>.remote, the upstream branch
- for the given branch. It tells 'git-fetch'/'git-pull' which
- branch to merge and can also affect 'git-push' (see push.default).
- When in branch <name>, it tells 'git-fetch' the default
+ for the given branch. It tells 'git fetch'/'git pull' which
+ branch to merge and can also affect 'git push' (see push.default).
+ When in branch <name>, it tells 'git fetch' the default
refspec to be marked for merging in FETCH_HEAD. The value is
handled like the remote part of a refspec, and must match a
ref which is fetched from the remote given by
"branch.<name>.remote".
- The merge information is used by 'git-pull' (which at first calls
- 'git-fetch') to lookup the default branch for merging. Without
- this option, 'git-pull' defaults to merge the first refspec fetched.
+ The merge information is used by 'git pull' (which at first calls
+ 'git fetch') to lookup the default branch for merging. Without
+ this option, 'git pull' defaults to merge the first refspec fetched.
Specify multiple values to get an octopus merge.
- If you wish to setup 'git-pull' so that it merges into <name> from
+ If you wish to setup 'git pull' so that it merges into <name> from
another branch in the local repository, you can point
branch.<name>.merge to the desired branch, and use the special setting
`.` (a period) for branch.<name>.remote.
`never`), never. When set to `true` or `auto`, use color only
when the output is written to the terminal. Defaults to `false`.
-color.grep.external::
- The string value of this variable is passed to an external 'grep'
- command as a command line option if match highlighting is turned
- on. If set to an empty string, no option is passed at all,
- turning off coloring for external 'grep' calls; this is the default.
- For GNU grep, set it to `--color=always` to highlight matches even
- when a pager is used.
-
color.grep.match::
Use customized color for matches. The value of this variable
may be specified as in color.branch.<slot>. It is passed using
colors only when the output is to the terminal. Defaults to false.
color.interactive.<slot>::
- Use customized color for 'git-add --interactive'
+ Use customized color for 'git add --interactive'
output. `<slot>` may be `prompt`, `header`, `help` or `error`, for
four distinct types of normal output from interactive
commands. The values of these variables may be specified as
terminal. When more specific variables of color.* are set, they always
take precedence over this setting. Defaults to false.
+commit.status::
+ A boolean to enable/disable inclusion of status information in the
+ commit message template when using an editor to prepare the commit
+ message. Defaults to true.
+
commit.template::
Specify a file to use as the template for new commit messages.
"{tilde}/" is expanded to the value of `$HOME` and "{tilde}user/" to the
specified user's home directory.
diff.autorefreshindex::
- When using 'git-diff' to compare with work tree
+ When using 'git diff' to compare with work tree
files, do not consider stat-only change as changed.
Instead, silently run `git update-index --refresh` to
update the cached stat information for paths whose
contents in the work tree match the contents in the
index. This option defaults to true. Note that this
- affects only 'git-diff' Porcelain, and not lower level
- 'diff' commands such as 'git-diff-files'.
+ affects only 'git diff' Porcelain, and not lower level
+ 'diff' commands such as 'git diff-files'.
diff.external::
If this config variable is set, diff generation is not
your files, you might want to use linkgit:gitattributes[5] instead.
diff.mnemonicprefix::
- If set, 'git-diff' uses a prefix pair that is different from the
+ If set, 'git diff' uses a prefix pair that is different from the
standard "a/" and "b/" depending on what is being compared. When
this configuration is in effect, reverse diff output also swaps
the order of the prefixes:
-'git-diff';;
+`git diff`;;
compares the (i)ndex and the (w)ork tree;
-'git-diff HEAD';;
+`git diff HEAD`;;
compares a (c)ommit and the (w)ork tree;
-'git diff --cached';;
+`git diff --cached`;;
compares a (c)ommit and the (i)ndex;
-'git-diff HEAD:file1 file2';;
+`git diff HEAD:file1 file2`;;
compares an (o)bject and a (w)ork tree entity;
-'git diff --no-index a b';;
+`git diff --no-index a b`;;
compares two non-git things (1) and (2).
diff.renameLimit::
The number of files to consider when performing the copy/rename
- detection; equivalent to the 'git-diff' option '-l'.
+ detection; equivalent to the 'git diff' option '-l'.
diff.renames::
Tells git to detect renames. If set to any boolean value, it
linkgit:git-whatchanged[1].
format.thread::
- The default threading style for 'git-format-patch'. Can be
+ The default threading style for 'git format-patch'. Can be
a boolean value, or `shallow` or `deep`. `shallow` threading
makes every mail a reply to the head of the series,
where the head is chosen from the cover letter, the
gc.aggressiveWindow::
The window size parameter used in the delta compression
- algorithm used by 'git-gc --aggressive'. This defaults
+ algorithm used by 'git gc --aggressive'. This defaults
to 10.
gc.auto::
Running `git pack-refs` in a repository renders it
unclonable by Git versions prior to 1.5.1.2 over dumb
transports such as HTTP. This variable determines whether
- 'git gc' runs `git pack-refs`. This can be set to "nobare"
+ 'git gc' runs `git pack-refs`. This can be set to `nobare`
to enable it within all non-bare repos or it can be set to a
boolean value. The default is `true`.
gc.pruneexpire::
- When 'git-gc' is run, it will call 'prune --expire 2.weeks.ago'.
+ When 'git gc' is run, it will call 'prune --expire 2.weeks.ago'.
Override the grace period with this config variable. The value
"now" may be used to disable this grace period and always prune
unreachable objects immediately.
gc.reflogexpire::
- 'git-reflog expire' removes reflog entries older than
+ 'git reflog expire' removes reflog entries older than
this time; defaults to 90 days.
gc.reflogexpireunreachable::
- 'git-reflog expire' removes reflog entries older than
+ 'git reflog expire' removes reflog entries older than
this time and are not reachable from the current tip;
defaults to 30 days.
gc.rerereresolved::
Records of conflicted merge you resolved earlier are
- kept for this many days when 'git-rerere gc' is run.
+ kept for this many days when 'git rerere gc' is run.
The default is 60 days. See linkgit:git-rerere[1].
gc.rerereunresolved::
Records of conflicted merge you have not resolved are
- kept for this many days when 'git-rerere gc' is run.
+ kept for this many days when 'git rerere gc' is run.
The default is 15 days. See linkgit:git-rerere[1].
gitcvs.commitmsgannotation::
off.
gui.fastcopyblame::
- If true, 'git gui blame' uses '-C' instead of '-C -C' for original
+ If true, 'git gui blame' uses `-C` instead of `-C -C` for original
location detection. It makes blame significantly faster on huge
repositories at the expense of less thorough copy detection.
How many HTTP requests to launch in parallel. Can be overridden
by the 'GIT_HTTP_MAX_REQUESTS' environment variable. Default is 5.
+http.minSessions::
+ The number of curl sessions (counted across slots) to be kept across
+ requests. They will not be ended with curl_easy_cleanup() until
+ http_cleanup() is invoked. If USE_CURL_MULTI is not defined, this
+ value will be capped at 1. Defaults to 1.
+
http.postBuffer::
Maximum size in bytes of the buffer used by smart HTTP
transports when POSTing data to the remote system.
i18n.logOutputEncoding::
Character encoding the commit messages are converted to when
- running 'git-log' and friends.
+ running 'git log' and friends.
imap::
The configuration variables in the 'imap' section are described
log.date::
Set default date-time mode for the log command. Setting log.date
- value is similar to using 'git-log'\'s --date option. The value is one of the
+ value is similar to using 'git log'\'s --date option. The value is one of the
following alternatives: {relative,local,default,iso,rfc,short}.
See linkgit:git-log[1].
Setting this value to \--no-tags disables automatic tag following when
fetching from remote <name>
+remote.<name>.vcs::
+ Setting this to a value <vcs> will cause git to interact with
+ the remote with the git-remote-<vcs> helper.
+
remotes.<group>::
The list of remotes which are fetched by "git remote update
<group>". See linkgit:git-remote[1].
diff --combined Documentation/git-fast-import.txt
index ff4022c15f2d178677447c8614a2914b104e2339,79c5f69d421399091b7ee6f71339505edf15ce0c..3f4380eb28fc543937fcb2d710acecb7af1d283d
This program is usually not what the end user wants to run directly.
Most end users want to use one of the existing frontend programs,
which parses a specific type of foreign source and feeds the contents
-stored there to 'git-fast-import'.
+stored there to 'git fast-import'.
fast-import reads a mixed command/data stream from standard input and
writes one or more packfiles directly into the current repository.
with the newly imported data.
The fast-import backend itself can import into an empty repository (one that
-has already been initialized by 'git-init') or incrementally
+has already been initialized by 'git init') or incrementally
update an existing populated repository. Whether or not incremental
imports are supported from a particular foreign source depends on
the frontend program in use.
importers may wish to lower this, such as to ensure the
resulting packfiles fit on CDs.
+ --big-file-threshold=<n>::
+ Maximum size of a blob that fast-import will attempt to
+ create a delta for, expressed in bytes. The default is 512m
+ (512 MiB). Some importers may wish to lower this on systems
+ with constrained memory.
+
--depth=<n>::
Maximum delta depth, for blob and tree deltification.
Default is 10.
set of marks. If a mark is defined to different values,
the last file wins.
+--relative-marks::
+ After specifying --relative-marks= the paths specified
+ with --import-marks= and --export-marks= are relative
+ to an internal directory in the current repository.
+ In git-fast-import this means that the paths are relative
+ to the .git/info/fast-import directory. However, other
+ importers may use a different location.
+
+--no-relative-marks::
+ Negates a previous --relative-marks. Allows for combining
+ relative and non-relative marks by interweaving
+ --(no-)-relative-marks= with the --(import|export)-marks=
+ options.
+
--export-pack-edges=<file>::
After creating a packfile, print a line of data to
<file> listing the filename of the packfile and the last
This information may be useful after importing projects
whose total object set exceeds the 4 GiB packfile limit,
as these commits can be used as edge points during calls
- to 'git-pack-objects'.
+ to 'git pack-objects'.
--quiet::
Disable all non-fatal output, making fast-import silent when it
Parallel Operation
------------------
-Like 'git-push' or 'git-fetch', imports handled by fast-import are safe to
+Like 'git push' or 'git fetch', imports handled by fast-import are safe to
run alongside parallel `git repack -a -d` or `git gc` invocations,
-or any other Git operation (including 'git-prune', as loose objects
+or any other Git operation (including 'git prune', as loose objects
are never used by fast-import).
fast-import does not lock the branch or tag refs it is actively importing.
+
An example value is ``Tue Feb 6 11:22:18 2007 -0500''. The Git
parser is accurate, but a little on the lenient side. It is the
-same parser used by 'git-am' when applying patches
+same parser used by 'git am' when applying patches
received from email.
+
Some malformed strings may be accepted as valid dates. In some of
This particular format is supplied as its short to implement and
may be useful to a process that wants to create a new commit
right now, without needing to use a working directory or
-'git-update-index'.
+'git update-index'.
+
If separate `author` and `committer` commands are used in a `commit`
the timestamps may not match, as the system clock will be polled
standard output. This command is optional and is not needed
to perform an import.
+`feature`::
+ Require that fast-import supports the specified feature, or
+ abort if it does not.
+
+`option`::
+ Specify any of the options listed under OPTIONS that do not
+ change stream semantic to suit the frontend's needs. This
+ command is optional and is not needed to perform an import.
+
`commit`
~~~~~~~~
Create or update a branch with a new commit, recording one logical
complete set of bytes which normally goes into such a signature.
If signing is required, create lightweight tags from within fast-import with
`reset`, then create the annotated versions of those tags offline
-with the standard 'git-tag' process.
+with the standard 'git tag' process.
`reset`
~~~~~~~
inform the reader when the `checkpoint` has been completed and it
can safely access the refs that fast-import updated.
+`feature`
+~~~~~~~~~
+Require that fast-import supports the specified feature, or abort if
+it does not.
+
+....
+ 'feature' SP <feature> LF
+....
+
+The <feature> part of the command may be any string matching
+^[a-zA-Z][a-zA-Z-]*$ and should be understood by fast-import.
+
+Feature work identical as their option counterparts with the
+exception of the import-marks feature, see below.
+
+The following features are currently supported:
+
+* date-format
+* import-marks
+* export-marks
+* relative-marks
+* no-relative-marks
+* force
+
+The import-marks behaves differently from when it is specified as
+commandline option in that only one "feature import-marks" is allowed
+per stream. Also, any --import-marks= specified on the commandline
+will override those from the stream (if any).
+
+`option`
+~~~~~~~~
+Processes the specified option so that git fast-import behaves in a
+way that suits the frontend's needs.
+Note that options specified by the frontend are overridden by any
+options the user may specify to git fast-import itself.
+
+....
+ 'option' SP <option> LF
+....
+
+The `<option>` part of the command may contain any of the options
+listed in the OPTIONS section that do not change import semantics,
+without the leading '--' and is treated in the same way.
+
+Option commands must be the first commands on the input (not counting
+feature commands), to give an option command after any non-option
+command is an error.
+
+The following commandline options change import semantics and may therefore
+not be passed as option:
+
+* date-format
+* import-marks
+* export-marks
+* force
+
Crash Reports
-------------
If fast-import is supplied invalid input it will terminate with a
When committing fixups, consider using `merge` to connect the
commit(s) which are supplying file revisions to the fixup branch.
-Doing so will allow tools such as 'git-blame' to track
+Doing so will allow tools such as 'git blame' to track
through the real commit history and properly annotate the source
files.
~~~~~~~~~~~~~~~~~~~~~~~~~
If you are repacking very old imported data (e.g. older than the
last year), consider expending some extra CPU time and supplying
-\--window=50 (or higher) when you run 'git-repack'.
+\--window=50 (or higher) when you run 'git repack'.
This will take longer, but will also produce a smaller packfile.
You only need to expend the effort once, and everyone using your
project will benefit from the smaller repository.
diff --combined contrib/completion/git-completion.bash
index 96517204105425ad15049d3bc713e3fbc8d86baa,1a762e88e7fcf3b36b3acdffa3e8ec18e3644e2e..7def62cb1d28bd9680e1c61aa74c9e7856d7dce3
fi
fi
- if [ -n "${1-}" ]; then
- printf "$1" "$c${b##refs/heads/}$w$i$s$u$r"
- else
- printf " (%s)" "$c${b##refs/heads/}$w$i$s$u$r"
- fi
+ local f="$w$i$s$u"
+ printf "${1:- (%s)}" "$c${b##refs/heads/}${f:+ $f}$r"
fi
}
read-tree) : plumbing;;
receive-pack) : plumbing;;
reflog) : plumbing;;
+ remote-*) : transport;;
repo-config) : deprecated;;
rerere) : plumbing;;
rev-list) : plumbing;;
init fetch clone rebase dcommit log find-rev
set-tree commit-diff info create-ignore propget
proplist show-ignore show-externals branch tag blame
- migrate
+ migrate mkdirs reset gc
"
local subcommand="$(__git_find_on_cmdline "$subcommands")"
if [ -z "$subcommand" ]; then
__gitcomp "--stdin $cmt_opts $fc_opts"
;;
create-ignore,--*|propget,--*|proplist,--*|show-ignore,--*|\
- show-externals,--*)
+ show-externals,--*|mkdirs,--*)
__gitcomp "--revision="
;;
log,--*)
--no-auth-cache --username=
"
;;
+ reset,--*)
+ __gitcomp "--revision= --parent"
+ ;;
*)
COMPREPLY=()
;;
diff --combined fast-import.c
index 901784fe911567752c872e5433f0295768654b5a,2105310c5412ac5afea99840080406340e26552b..ca210822dbe88ce82d6eaf9d435dd68561e90c9f
--- 1/fast-import.c
--- 2/fast-import.c
+++ b/fast-import.c
const char *name;
struct tree_entry branch_tree;
uintmax_t last_commit;
+ uintmax_t num_notes;
unsigned active : 1;
unsigned pack_id : PACK_ID_BITS;
unsigned char sha1[20];
/* Configured limits on output */
static unsigned long max_depth = 10;
static off_t max_packsize = (1LL << 32) - 1;
+ static uintmax_t big_file_threshold = 512 * 1024 * 1024;
static int force_update;
static int pack_compression_level = Z_DEFAULT_COMPRESSION;
static int pack_compression_seen;
static unsigned long branch_load_count;
static int failure;
static FILE *pack_edges;
+static unsigned int show_stats = 1;
+static int global_argc;
+static const char **global_argv;
/* Memory pools */
static size_t mem_pool_alloc = 2*1024*1024 - sizeof(struct mem_pool);
static struct object_entry_pool *blocks;
static struct object_entry *object_table[1 << 16];
static struct mark_set *marks;
-static const char *mark_file;
+static const char *export_marks_file;
+static const char *import_marks_file;
+static int import_marks_file_from_stream;
+static int relative_marks_paths;
/* Our last blob */
static struct last_object last_blob = { STRBUF_INIT, 0, 0, 0 };
static unsigned int cmd_save = 100;
static uintmax_t next_mark;
static struct strbuf new_data = STRBUF_INIT;
+static int seen_data_command;
+
+static void parse_argv(void);
static void write_branch_report(FILE *rpt, struct branch *b)
{
fputc('\n', rpt);
fputs("Marks\n", rpt);
fputs("-----\n", rpt);
- if (mark_file)
- fprintf(rpt, " exported to %s\n", mark_file);
+ if (export_marks_file)
+ fprintf(rpt, " exported to %s\n", export_marks_file);
else
dump_marks_helper(rpt, 0, marks);
b->table_next_branch = branch_table[hc];
b->branch_tree.versions[0].mode = S_IFDIR;
b->branch_tree.versions[1].mode = S_IFDIR;
+ b->num_notes = 0;
b->active = 0;
b->pack_id = MAX_PACK_ID;
branch_table[hc] = b;
static size_t encode_header(
enum object_type type,
- size_t size,
+ uintmax_t size,
unsigned char *hdr)
{
int n = 1;
return 0;
}
+ static void truncate_pack(off_t to)
+ {
+ if (ftruncate(pack_data->pack_fd, to)
+ || lseek(pack_data->pack_fd, to, SEEK_SET) != to)
+ die_errno("cannot truncate pack to skip duplicate");
+ pack_size = to;
+ }
+
+ static void stream_blob(uintmax_t len, unsigned char *sha1out, uintmax_t mark)
+ {
+ size_t in_sz = 64 * 1024, out_sz = 64 * 1024;
+ unsigned char *in_buf = xmalloc(in_sz);
+ unsigned char *out_buf = xmalloc(out_sz);
+ struct object_entry *e;
+ unsigned char sha1[20];
+ unsigned long hdrlen;
+ off_t offset;
+ git_SHA_CTX c;
+ z_stream s;
+ int status = Z_OK;
+
+ /* Determine if we should auto-checkpoint. */
+ if ((pack_size + 60 + len) > max_packsize
+ || (pack_size + 60 + len) < pack_size)
+ cycle_packfile();
+
+ offset = pack_size;
+
+ hdrlen = snprintf((char *)out_buf, out_sz, "blob %" PRIuMAX, len) + 1;
+ if (out_sz <= hdrlen)
+ die("impossibly large object header");
+
+ git_SHA1_Init(&c);
+ git_SHA1_Update(&c, out_buf, hdrlen);
+
+ memset(&s, 0, sizeof(s));
+ deflateInit(&s, pack_compression_level);
+
+ hdrlen = encode_header(OBJ_BLOB, len, out_buf);
+ if (out_sz <= hdrlen)
+ die("impossibly large object header");
+
+ s.next_out = out_buf + hdrlen;
+ s.avail_out = out_sz - hdrlen;
+
+ while (status != Z_STREAM_END) {
+ if (0 < len && !s.avail_in) {
+ size_t cnt = in_sz < len ? in_sz : (size_t)len;
+ size_t n = fread(in_buf, 1, cnt, stdin);
+ if (!n && feof(stdin))
+ die("EOF in data (%" PRIuMAX " bytes remaining)", len);
+
+ git_SHA1_Update(&c, in_buf, n);
+ s.next_in = in_buf;
+ s.avail_in = n;
+ len -= n;
+ }
+
+ status = deflate(&s, len ? 0 : Z_FINISH);
+
+ if (!s.avail_out || status == Z_STREAM_END) {
+ size_t n = s.next_out - out_buf;
+ write_or_die(pack_data->pack_fd, out_buf, n);
+ pack_size += n;
+ s.next_out = out_buf;
+ s.avail_out = out_sz;
+ }
+
+ switch (status) {
+ case Z_OK:
+ case Z_BUF_ERROR:
+ case Z_STREAM_END:
+ continue;
+ default:
+ die("unexpected deflate failure: %d", status);
+ }
+ }
+ deflateEnd(&s);
+ git_SHA1_Final(sha1, &c);
+
+ if (sha1out)
+ hashcpy(sha1out, sha1);
+
+ e = insert_object(sha1);
+
+ if (mark)
+ insert_mark(mark, e);
+
+ if (e->offset) {
+ duplicate_count_by_type[OBJ_BLOB]++;
+ truncate_pack(offset);
+
+ } else if (find_sha1_pack(sha1, packed_git)) {
+ e->type = OBJ_BLOB;
+ e->pack_id = MAX_PACK_ID;
+ e->offset = 1; /* just not zero! */
+ duplicate_count_by_type[OBJ_BLOB]++;
+ truncate_pack(offset);
+
+ } else {
+ e->depth = 0;
+ e->type = OBJ_BLOB;
+ e->pack_id = pack_id;
+ e->offset = offset;
+ object_count++;
+ object_count_by_type[OBJ_BLOB]++;
+ }
+
+ free(in_buf);
+ free(out_buf);
+ }
+
/* All calls must be guarded by find_object() or find_mark() to
* ensure the 'struct object_entry' passed was written by this
* process instance. We unpack the entry by the offset, avoiding
int mark_fd;
FILE *f;
- if (!mark_file)
+ if (!export_marks_file)
return;
- mark_fd = hold_lock_file_for_update(&mark_lock, mark_file, 0);
+ mark_fd = hold_lock_file_for_update(&mark_lock, export_marks_file, 0);
if (mark_fd < 0) {
failure |= error("Unable to write marks file %s: %s",
- mark_file, strerror(errno));
+ export_marks_file, strerror(errno));
return;
}
int saved_errno = errno;
rollback_lock_file(&mark_lock);
failure |= error("Unable to write marks file %s: %s",
- mark_file, strerror(saved_errno));
+ export_marks_file, strerror(saved_errno));
return;
}
int saved_errno = errno;
rollback_lock_file(&mark_lock);
failure |= error("Unable to write marks file %s: %s",
- mark_file, strerror(saved_errno));
+ export_marks_file, strerror(saved_errno));
return;
}
int saved_errno = errno;
rollback_lock_file(&mark_lock);
failure |= error("Unable to commit marks file %s: %s",
- mark_file, strerror(saved_errno));
+ export_marks_file, strerror(saved_errno));
return;
}
}
+static void read_marks(void)
+{
+ char line[512];
+ FILE *f = fopen(import_marks_file, "r");
+ if (!f)
+ die_errno("cannot read '%s'", import_marks_file);
+ while (fgets(line, sizeof(line), f)) {
+ uintmax_t mark;
+ char *end;
+ unsigned char sha1[20];
+ struct object_entry *e;
+
+ end = strchr(line, '\n');
+ if (line[0] != ':' || !end)
+ die("corrupt mark line: %s", line);
+ *end = 0;
+ mark = strtoumax(line + 1, &end, 10);
+ if (!mark || end == line + 1
+ || *end != ' ' || get_sha1(end + 1, sha1))
+ die("corrupt mark line: %s", line);
+ e = find_object(sha1);
+ if (!e) {
+ enum object_type type = sha1_object_info(sha1, NULL);
+ if (type < 0)
+ die("object not found: %s", sha1_to_hex(sha1));
+ e = insert_object(sha1);
+ e->type = type;
+ e->pack_id = MAX_PACK_ID;
+ e->offset = 1; /* just not zero! */
+ }
+ insert_mark(mark, e);
+ }
+ fclose(f);
+}
+
+
static int read_next_command(void)
{
static int stdin_eof = 0;
if (stdin_eof)
return EOF;
+ if (!seen_data_command
+ && prefixcmp(command_buf.buf, "feature ")
+ && prefixcmp(command_buf.buf, "option ")) {
+ parse_argv();
+ }
+
rc = rc_free;
if (rc)
rc_free = rc->next;
next_mark = 0;
}
- static void parse_data(struct strbuf *sb)
+ static int parse_data(struct strbuf *sb, uintmax_t limit, uintmax_t *len_res)
{
strbuf_reset(sb);
free(term);
}
else {
- size_t n = 0, length;
+ uintmax_t len = strtoumax(command_buf.buf + 5, NULL, 10);
+ size_t n = 0, length = (size_t)len;
- length = strtoul(command_buf.buf + 5, NULL, 10);
+ if (limit && limit < len) {
+ *len_res = len;
+ return 0;
+ }
+ if (length < len)
+ die("data is too large to use in this context");
while (n < length) {
size_t s = strbuf_fread(sb, length - n, stdin);
}
skip_optional_lf();
+ return 1;
}
static int validate_raw_date(const char *src, char *result, int maxlen)
return ident;
}
- static void parse_new_blob(void)
+ static void parse_and_store_blob(
+ struct last_object *last,
+ unsigned char *sha1out,
+ uintmax_t mark)
{
static struct strbuf buf = STRBUF_INIT;
+ uintmax_t len;
+ if (parse_data(&buf, big_file_threshold, &len))
+ store_object(OBJ_BLOB, &buf, last, sha1out, mark);
+ else {
+ if (last) {
+ strbuf_release(&last->data);
+ last->offset = 0;
+ last->depth = 0;
+ }
+ stream_blob(len, sha1out, mark);
+ skip_optional_lf();
+ }
+ }
+
+ static void parse_new_blob(void)
+ {
read_next_command();
parse_mark();
- parse_data(&buf);
- store_object(OBJ_BLOB, &buf, &last_blob, NULL, next_mark);
+ parse_and_store_blob(&last_blob, NULL, next_mark);
}
static void unload_one_branch(void)
}
}
+static unsigned char convert_num_notes_to_fanout(uintmax_t num_notes)
+{
+ unsigned char fanout = 0;
+ while ((num_notes >>= 8))
+ fanout++;
+ return fanout;
+}
+
+static void construct_path_with_fanout(const char *hex_sha1,
+ unsigned char fanout, char *path)
+{
+ unsigned int i = 0, j = 0;
+ if (fanout >= 20)
+ die("Too large fanout (%u)", fanout);
+ while (fanout) {
+ path[i++] = hex_sha1[j++];
+ path[i++] = hex_sha1[j++];
+ path[i++] = '/';
+ fanout--;
+ }
+ memcpy(path + i, hex_sha1 + j, 40 - j);
+ path[i + 40 - j] = '\0';
+}
+
+static uintmax_t do_change_note_fanout(
+ struct tree_entry *orig_root, struct tree_entry *root,
+ char *hex_sha1, unsigned int hex_sha1_len,
+ char *fullpath, unsigned int fullpath_len,
+ unsigned char fanout)
+{
+ struct tree_content *t = root->tree;
+ struct tree_entry *e, leaf;
+ unsigned int i, tmp_hex_sha1_len, tmp_fullpath_len;
+ uintmax_t num_notes = 0;
+ unsigned char sha1[20];
+ char realpath[60];
+
+ for (i = 0; t && i < t->entry_count; i++) {
+ e = t->entries[i];
+ tmp_hex_sha1_len = hex_sha1_len + e->name->str_len;
+ tmp_fullpath_len = fullpath_len;
+
+ /*
+ * We're interested in EITHER existing note entries (entries
+ * with exactly 40 hex chars in path, not including directory
+ * separators), OR directory entries that may contain note
+ * entries (with < 40 hex chars in path).
+ * Also, each path component in a note entry must be a multiple
+ * of 2 chars.
+ */
+ if (!e->versions[1].mode ||
+ tmp_hex_sha1_len > 40 ||
+ e->name->str_len % 2)
+ continue;
+
+ /* This _may_ be a note entry, or a subdir containing notes */
+ memcpy(hex_sha1 + hex_sha1_len, e->name->str_dat,
+ e->name->str_len);
+ if (tmp_fullpath_len)
+ fullpath[tmp_fullpath_len++] = '/';
+ memcpy(fullpath + tmp_fullpath_len, e->name->str_dat,
+ e->name->str_len);
+ tmp_fullpath_len += e->name->str_len;
+ fullpath[tmp_fullpath_len] = '\0';
+
+ if (tmp_hex_sha1_len == 40 && !get_sha1_hex(hex_sha1, sha1)) {
+ /* This is a note entry */
+ construct_path_with_fanout(hex_sha1, fanout, realpath);
+ if (!strcmp(fullpath, realpath)) {
+ /* Note entry is in correct location */
+ num_notes++;
+ continue;
+ }
+
+ /* Rename fullpath to realpath */
+ if (!tree_content_remove(orig_root, fullpath, &leaf))
+ die("Failed to remove path %s", fullpath);
+ tree_content_set(orig_root, realpath,
+ leaf.versions[1].sha1,
+ leaf.versions[1].mode,
+ leaf.tree);
+ } else if (S_ISDIR(e->versions[1].mode)) {
+ /* This is a subdir that may contain note entries */
+ if (!e->tree)
+ load_tree(e);
+ num_notes += do_change_note_fanout(orig_root, e,
+ hex_sha1, tmp_hex_sha1_len,
+ fullpath, tmp_fullpath_len, fanout);
+ }
+
+ /* The above may have reallocated the current tree_content */
+ t = root->tree;
+ }
+ return num_notes;
+}
+
+static uintmax_t change_note_fanout(struct tree_entry *root,
+ unsigned char fanout)
+{
+ char hex_sha1[40], path[60];
+ return do_change_note_fanout(root, root, hex_sha1, 0, path, 0, fanout);
+}
+
static void file_change_m(struct branch *b)
{
const char *p = command_buf.buf + 2;
* another repository.
*/
} else if (inline_data) {
- static struct strbuf buf = STRBUF_INIT;
-
if (p != uq.buf) {
strbuf_addstr(&uq, p);
p = uq.buf;
}
read_next_command();
- parse_data(&buf);
- store_object(OBJ_BLOB, &buf, &last_blob, sha1, 0);
+ parse_and_store_blob(&last_blob, sha1, 0);
} else if (oe) {
if (oe->type != OBJ_BLOB)
die("Not a blob (actually a %s): %s",
leaf.tree);
}
-static void note_change_n(struct branch *b)
+static void note_change_n(struct branch *b, unsigned char old_fanout)
{
const char *p = command_buf.buf + 2;
static struct strbuf uq = STRBUF_INIT;
struct object_entry *oe = oe;
struct branch *s;
unsigned char sha1[20], commit_sha1[20];
+ char path[60];
uint16_t inline_data = 0;
+ unsigned char new_fanout;
/* <dataref> or 'inline' */
if (*p == ':') {
die("Invalid ref name or SHA1 expression: %s", p);
if (inline_data) {
- static struct strbuf buf = STRBUF_INIT;
-
if (p != uq.buf) {
strbuf_addstr(&uq, p);
p = uq.buf;
}
read_next_command();
- parse_data(&buf);
- store_object(OBJ_BLOB, &buf, &last_blob, sha1, 0);
+ parse_and_store_blob(&last_blob, sha1, 0);
} else if (oe) {
if (oe->type != OBJ_BLOB)
die("Not a blob (actually a %s): %s",
typename(oe->type), command_buf.buf);
- } else {
+ } else if (!is_null_sha1(sha1)) {
enum object_type type = sha1_object_info(sha1, NULL);
if (type < 0)
die("Blob not found: %s", command_buf.buf);
typename(type), command_buf.buf);
}
- tree_content_set(&b->branch_tree, sha1_to_hex(commit_sha1), sha1,
- S_IFREG | 0644, NULL);
+ construct_path_with_fanout(sha1_to_hex(commit_sha1), old_fanout, path);
+ if (tree_content_remove(&b->branch_tree, path, NULL))
+ b->num_notes--;
+
+ if (is_null_sha1(sha1))
+ return; /* nothing to insert */
+
+ b->num_notes++;
+ new_fanout = convert_num_notes_to_fanout(b->num_notes);
+ construct_path_with_fanout(sha1_to_hex(commit_sha1), new_fanout, path);
+ tree_content_set(&b->branch_tree, path, sha1, S_IFREG | 0644, NULL);
}
static void file_change_deleteall(struct branch *b)
hashclr(b->branch_tree.versions[0].sha1);
hashclr(b->branch_tree.versions[1].sha1);
load_tree(&b->branch_tree);
+ b->num_notes = 0;
}
static void parse_from_commit(struct branch *b, char *buf, unsigned long size)
char *committer = NULL;
struct hash_list *merge_list = NULL;
unsigned int merge_count;
+ unsigned char prev_fanout, new_fanout;
/* Obtain the branch name from the rest of our command */
sp = strchr(command_buf.buf, ' ') + 1;
}
if (!committer)
die("Expected committer but didn't get one");
- parse_data(&msg);
+ parse_data(&msg, 0, NULL);
read_next_command();
parse_from(b);
merge_list = parse_merge(&merge_count);
load_branch(b);
}
+ prev_fanout = convert_num_notes_to_fanout(b->num_notes);
+
/* file_change* */
while (command_buf.len > 0) {
if (!prefixcmp(command_buf.buf, "M "))
else if (!prefixcmp(command_buf.buf, "C "))
file_change_cr(b, 0);
else if (!prefixcmp(command_buf.buf, "N "))
- note_change_n(b);
+ note_change_n(b, prev_fanout);
else if (!strcmp("deleteall", command_buf.buf))
file_change_deleteall(b);
else {
break;
}
+ new_fanout = convert_num_notes_to_fanout(b->num_notes);
+ if (new_fanout != prev_fanout)
+ b->num_notes = change_note_fanout(&b->branch_tree, new_fanout);
+
/* build the tree and the commit */
store_tree(&b->branch_tree);
hashcpy(b->branch_tree.versions[0].sha1,
tagger = NULL;
/* tag payload/message */
- parse_data(&msg);
+ parse_data(&msg, 0, NULL);
/* build the tag object */
strbuf_reset(&new_data);
skip_optional_lf();
}
-static void import_marks(const char *input_file)
+static char* make_fast_import_path(const char *path)
{
- char line[512];
- FILE *f = fopen(input_file, "r");
- if (!f)
- die_errno("cannot read '%s'", input_file);
- while (fgets(line, sizeof(line), f)) {
- uintmax_t mark;
- char *end;
- unsigned char sha1[20];
- struct object_entry *e;
+ struct strbuf abs_path = STRBUF_INIT;
- end = strchr(line, '\n');
- if (line[0] != ':' || !end)
- die("corrupt mark line: %s", line);
- *end = 0;
- mark = strtoumax(line + 1, &end, 10);
- if (!mark || end == line + 1
- || *end != ' ' || get_sha1(end + 1, sha1))
- die("corrupt mark line: %s", line);
- e = find_object(sha1);
- if (!e) {
- enum object_type type = sha1_object_info(sha1, NULL);
- if (type < 0)
- die("object not found: %s", sha1_to_hex(sha1));
- e = insert_object(sha1);
- e->type = type;
- e->pack_id = MAX_PACK_ID;
- e->offset = 1; /* just not zero! */
- }
- insert_mark(mark, e);
+ if (!relative_marks_paths || is_absolute_path(path))
+ return xstrdup(path);
+ strbuf_addf(&abs_path, "%s/info/fast-import/%s", get_git_dir(), path);
+ return strbuf_detach(&abs_path, NULL);
+}
+
+static void option_import_marks(const char *marks, int from_stream)
+{
+ if (import_marks_file) {
+ if (from_stream)
+ die("Only one import-marks command allowed per stream");
+
+ /* read previous mark file */
+ if(!import_marks_file_from_stream)
+ read_marks();
}
- fclose(f);
+
+ import_marks_file = make_fast_import_path(marks);
+ import_marks_file_from_stream = from_stream;
+}
+
+static void option_date_format(const char *fmt)
+{
+ if (!strcmp(fmt, "raw"))
+ whenspec = WHENSPEC_RAW;
+ else if (!strcmp(fmt, "rfc2822"))
+ whenspec = WHENSPEC_RFC2822;
+ else if (!strcmp(fmt, "now"))
+ whenspec = WHENSPEC_NOW;
+ else
+ die("unknown --date-format argument %s", fmt);
+}
+
+static void option_max_pack_size(const char *packsize)
+{
+ max_packsize = strtoumax(packsize, NULL, 0) * 1024 * 1024;
+}
+
+static void option_depth(const char *depth)
+{
+ max_depth = strtoul(depth, NULL, 0);
+ if (max_depth > MAX_DEPTH)
+ die("--depth cannot exceed %u", MAX_DEPTH);
+}
+
+static void option_active_branches(const char *branches)
+{
+ max_active_branches = strtoul(branches, NULL, 0);
+}
+
+static void option_export_marks(const char *marks)
+{
+ export_marks_file = make_fast_import_path(marks);
+}
+
+static void option_export_pack_edges(const char *edges)
+{
+ if (pack_edges)
+ fclose(pack_edges);
+ pack_edges = fopen(edges, "a");
+ if (!pack_edges)
+ die_errno("Cannot open '%s'", edges);
+}
+
+static int parse_one_option(const char *option)
+{
+ if (!prefixcmp(option, "max-pack-size=")) {
+ option_max_pack_size(option + 14);
++ } else if (!prefixcmp(option, "big-file-threshold=")) {
++ big_file_threshold = strtoumax(option + 19, NULL, 0) * 1024 * 1024;
+ } else if (!prefixcmp(option, "depth=")) {
+ option_depth(option + 6);
+ } else if (!prefixcmp(option, "active-branches=")) {
+ option_active_branches(option + 16);
+ } else if (!prefixcmp(option, "export-pack-edges=")) {
+ option_export_pack_edges(option + 18);
+ } else if (!prefixcmp(option, "quiet")) {
+ show_stats = 0;
+ } else if (!prefixcmp(option, "stats")) {
+ show_stats = 1;
+ } else {
+ return 0;
+ }
+
+ return 1;
+}
+
+static int parse_one_feature(const char *feature, int from_stream)
+{
+ if (!prefixcmp(feature, "date-format=")) {
+ option_date_format(feature + 12);
+ } else if (!prefixcmp(feature, "import-marks=")) {
+ option_import_marks(feature + 13, from_stream);
+ } else if (!prefixcmp(feature, "export-marks=")) {
+ option_export_marks(feature + 13);
+ } else if (!prefixcmp(feature, "relative-marks")) {
+ relative_marks_paths = 1;
+ } else if (!prefixcmp(feature, "no-relative-marks")) {
+ relative_marks_paths = 0;
+ } else if (!prefixcmp(feature, "force")) {
+ force_update = 1;
+ } else {
+ return 0;
+ }
+
+ return 1;
+}
+
+static void parse_feature(void)
+{
+ char *feature = command_buf.buf + 8;
+
+ if (seen_data_command)
+ die("Got feature command '%s' after data command", feature);
+
+ if (parse_one_feature(feature, 1))
+ return;
+
+ die("This version of fast-import does not support feature %s.", feature);
+}
+
+static void parse_option(void)
+{
+ char *option = command_buf.buf + 11;
+
+ if (seen_data_command)
+ die("Got option command '%s' after data command", option);
+
+ if (parse_one_option(option))
+ return;
+
+ die("This version of fast-import does not support option: %s", option);
}
static int git_pack_config(const char *k, const char *v, void *cb)
pack_compression_seen = 1;
return 0;
}
+ if (!strcmp(k, "core.bigfilethreshold")) {
+ long n = git_config_int(k, v);
+ big_file_threshold = 0 < n ? n : 0;
+ }
return git_default_config(k, v, cb);
}
static const char fast_import_usage[] =
- "git fast-import [--date-format=f] [--max-pack-size=n] [--depth=n] [--active-branches=n] [--export-marks=marks.file]";
+ "git fast-import [--date-format=f] [--max-pack-size=n] [--big-file-threshold=n] [--depth=n] [--active-branches=n] [--export-marks=marks.file]";
+static void parse_argv(void)
+{
+ unsigned int i;
+
+ for (i = 1; i < global_argc; i++) {
+ const char *a = global_argv[i];
+
+ if (*a != '-' || !strcmp(a, "--"))
+ break;
+
+ if (parse_one_option(a + 2))
+ continue;
+
+ if (parse_one_feature(a + 2, 0))
+ continue;
+
+ die("unknown option %s", a);
+ }
+ if (i != global_argc)
+ usage(fast_import_usage);
+
+ seen_data_command = 1;
+ if (import_marks_file)
+ read_marks();
+}
+
int main(int argc, const char **argv)
{
- unsigned int i, show_stats = 1;
+ unsigned int i;
git_extract_argv0_path(argv[0]);
avail_tree_table = xcalloc(avail_tree_table_sz, sizeof(struct avail_tree_content*));
marks = pool_calloc(1, sizeof(struct mark_set));
- for (i = 1; i < argc; i++) {
- const char *a = argv[i];
-
- if (*a != '-' || !strcmp(a, "--"))
- break;
- else if (!prefixcmp(a, "--date-format=")) {
- const char *fmt = a + 14;
- if (!strcmp(fmt, "raw"))
- whenspec = WHENSPEC_RAW;
- else if (!strcmp(fmt, "rfc2822"))
- whenspec = WHENSPEC_RFC2822;
- else if (!strcmp(fmt, "now"))
- whenspec = WHENSPEC_NOW;
- else
- die("unknown --date-format argument %s", fmt);
- }
- else if (!prefixcmp(a, "--max-pack-size="))
- max_packsize = strtoumax(a + 16, NULL, 0) * 1024 * 1024;
- else if (!prefixcmp(a, "--big-file-threshold=")) {
- unsigned long v;
- if (!git_parse_ulong(a + 21, &v))
- usage(fast_import_usage);
- big_file_threshold = v;
- } else if (!prefixcmp(a, "--depth=")) {
- max_depth = strtoul(a + 8, NULL, 0);
- if (max_depth > MAX_DEPTH)
- die("--depth cannot exceed %u", MAX_DEPTH);
- }
- else if (!prefixcmp(a, "--active-branches="))
- max_active_branches = strtoul(a + 18, NULL, 0);
- else if (!prefixcmp(a, "--import-marks="))
- import_marks(a + 15);
- else if (!prefixcmp(a, "--export-marks="))
- mark_file = a + 15;
- else if (!prefixcmp(a, "--export-pack-edges=")) {
- if (pack_edges)
- fclose(pack_edges);
- pack_edges = fopen(a + 20, "a");
- if (!pack_edges)
- die_errno("Cannot open '%s'", a + 20);
- } else if (!strcmp(a, "--force"))
- force_update = 1;
- else if (!strcmp(a, "--quiet"))
- show_stats = 0;
- else if (!strcmp(a, "--stats"))
- show_stats = 1;
- else
- die("unknown option %s", a);
- }
- if (i != argc)
- usage(fast_import_usage);
+ global_argc = argc;
+ global_argv = argv;
rc_free = pool_alloc(cmd_save * sizeof(*rc_free));
for (i = 0; i < (cmd_save - 1); i++)
parse_checkpoint();
else if (!prefixcmp(command_buf.buf, "progress "))
parse_progress();
+ else if (!prefixcmp(command_buf.buf, "feature "))
+ parse_feature();
+ else if (!prefixcmp(command_buf.buf, "option git "))
+ parse_option();
+ else if (!prefixcmp(command_buf.buf, "option "))
+ /* ignore non-git options*/;
else
die("Unsupported command: %s", command_buf.buf);
}
+
+ /* argv hasn't been parsed yet, do so */
+ if (!seen_data_command)
+ parse_argv();
+
end_packfile();
dump_branches();
diff --combined t/t9300-fast-import.sh
index 60d6f5d1ba7d5421c4c364e070e35bdb024b3b4a,513db86ad28227b5264b16a659802c5c59bbd0ea..131f03298809ad193cc75ab77deda6daaf713d1f
+++ b/t/t9300-fast-import.sh
### series Q (notes)
###
-note1_data="Note for the first commit"
-note2_data="Note for the second commit"
-note3_data="Note for the third commit"
+note1_data="The first note for the first commit"
+note2_data="The first note for the second commit"
+note3_data="The first note for the third commit"
+note1b_data="The second note for the first commit"
+note1c_data="The third note for the first commit"
+note2b_data="The second note for the second commit"
test_tick
cat >input <<INPUT_END
$note3_data
EOF
+commit refs/notes/foobar
+mark :10
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+notes (:10)
+COMMIT
+
+N inline :3
+data <<EOF
+$note1b_data
+EOF
+
+commit refs/notes/foobar2
+mark :11
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+notes (:11)
+COMMIT
+
+N inline :3
+data <<EOF
+$note1c_data
+EOF
+
+commit refs/notes/foobar
+mark :12
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+data <<COMMIT
+notes (:12)
+COMMIT
+
+deleteall
+N inline :5
+data <<EOF
+$note2b_data
+EOF
+
INPUT_END
+
test_expect_success \
'Q: commit notes' \
'git fast-import <input &&
notes (:9)
EOF
test_expect_success \
- 'Q: verify notes commit' \
- 'git cat-file commit refs/notes/foobar | sed 1d >actual &&
+ 'Q: verify first notes commit' \
+ 'git cat-file commit refs/notes/foobar~2 | sed 1d >actual &&
test_cmp expect actual'
cat >expect.unsorted <<EOF
EOF
cat expect.unsorted | sort >expect
test_expect_success \
- 'Q: verify notes tree' \
- 'git cat-file -p refs/notes/foobar^{tree} | sed "s/ [0-9a-f]* / /" >actual &&
+ 'Q: verify first notes tree' \
+ 'git cat-file -p refs/notes/foobar~2^{tree} | sed "s/ [0-9a-f]* / /" >actual &&
test_cmp expect actual'
echo "$note1_data" >expect
test_expect_success \
- 'Q: verify note for first commit' \
- 'git cat-file blob refs/notes/foobar:$commit1 >actual && test_cmp expect actual'
+ 'Q: verify first note for first commit' \
+ 'git cat-file blob refs/notes/foobar~2:$commit1 >actual && test_cmp expect actual'
echo "$note2_data" >expect
test_expect_success \
- 'Q: verify note for second commit' \
- 'git cat-file blob refs/notes/foobar:$commit2 >actual && test_cmp expect actual'
+ 'Q: verify first note for second commit' \
+ 'git cat-file blob refs/notes/foobar~2:$commit2 >actual && test_cmp expect actual'
echo "$note3_data" >expect
test_expect_success \
- 'Q: verify note for third commit' \
- 'git cat-file blob refs/notes/foobar:$commit3 >actual && test_cmp expect actual'
+ 'Q: verify first note for third commit' \
+ 'git cat-file blob refs/notes/foobar~2:$commit3 >actual && test_cmp expect actual'
+
+cat >expect <<EOF
+parent `git rev-parse --verify refs/notes/foobar~2`
+author $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+
+notes (:10)
+EOF
+test_expect_success \
+ 'Q: verify second notes commit' \
+ 'git cat-file commit refs/notes/foobar^ | sed 1d >actual &&
+ test_cmp expect actual'
+
+cat >expect.unsorted <<EOF
+100644 blob $commit1
+100644 blob $commit2
+100644 blob $commit3
+EOF
+cat expect.unsorted | sort >expect
+test_expect_success \
+ 'Q: verify second notes tree' \
+ 'git cat-file -p refs/notes/foobar^^{tree} | sed "s/ [0-9a-f]* / /" >actual &&
+ test_cmp expect actual'
+
+echo "$note1b_data" >expect
+test_expect_success \
+ 'Q: verify second note for first commit' \
+ 'git cat-file blob refs/notes/foobar^:$commit1 >actual && test_cmp expect actual'
+
+echo "$note2_data" >expect
+test_expect_success \
+ 'Q: verify first note for second commit' \
+ 'git cat-file blob refs/notes/foobar^:$commit2 >actual && test_cmp expect actual'
+
+echo "$note3_data" >expect
+test_expect_success \
+ 'Q: verify first note for third commit' \
+ 'git cat-file blob refs/notes/foobar^:$commit3 >actual && test_cmp expect actual'
+
+cat >expect <<EOF
+author $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+
+notes (:11)
+EOF
+test_expect_success \
+ 'Q: verify third notes commit' \
+ 'git cat-file commit refs/notes/foobar2 | sed 1d >actual &&
+ test_cmp expect actual'
+
+cat >expect.unsorted <<EOF
+100644 blob $commit1
+EOF
+cat expect.unsorted | sort >expect
+test_expect_success \
+ 'Q: verify third notes tree' \
+ 'git cat-file -p refs/notes/foobar2^{tree} | sed "s/ [0-9a-f]* / /" >actual &&
+ test_cmp expect actual'
+
+echo "$note1c_data" >expect
+test_expect_success \
+ 'Q: verify third note for first commit' \
+ 'git cat-file blob refs/notes/foobar2:$commit1 >actual && test_cmp expect actual'
+
+cat >expect <<EOF
+parent `git rev-parse --verify refs/notes/foobar^`
+author $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+
+notes (:12)
+EOF
+test_expect_success \
+ 'Q: verify fourth notes commit' \
+ 'git cat-file commit refs/notes/foobar | sed 1d >actual &&
+ test_cmp expect actual'
+
+cat >expect.unsorted <<EOF
+100644 blob $commit2
+EOF
+cat expect.unsorted | sort >expect
+test_expect_success \
+ 'Q: verify fourth notes tree' \
+ 'git cat-file -p refs/notes/foobar^{tree} | sed "s/ [0-9a-f]* / /" >actual &&
+ test_cmp expect actual'
+
+echo "$note2b_data" >expect
+test_expect_success \
+ 'Q: verify second note for second commit' \
+ 'git cat-file blob refs/notes/foobar:$commit2 >actual && test_cmp expect actual'
+
+###
+### series R (feature and option)
+###
+
+cat >input <<EOF
+feature no-such-feature-exists
+EOF
+
+test_expect_success 'R: abort on unsupported feature' '
+ test_must_fail git fast-import <input
+'
+
+cat >input <<EOF
+feature date-format=now
+EOF
+
+test_expect_success 'R: supported feature is accepted' '
+ git fast-import <input
+'
+
+cat >input << EOF
+blob
+data 3
+hi
+feature date-format=now
+EOF
+
+test_expect_success 'R: abort on receiving feature after data command' '
+ test_must_fail git fast-import <input
+'
+
+cat >input << EOF
+feature import-marks=git.marks
+feature import-marks=git2.marks
+EOF
+
+test_expect_success 'R: only one import-marks feature allowed per stream' '
+ test_must_fail git fast-import <input
+'
+
+cat >input << EOF
+feature export-marks=git.marks
+blob
+mark :1
+data 3
+hi
+
+EOF
+
+test_expect_success \
+ 'R: export-marks feature results in a marks file being created' \
+ 'cat input | git fast-import &&
+ grep :1 git.marks'
+
+test_expect_success \
+ 'R: export-marks options can be overriden by commandline options' \
+ 'cat input | git fast-import --export-marks=other.marks &&
+ grep :1 other.marks'
+
+cat >input << EOF
+feature import-marks=marks.out
+feature export-marks=marks.new
+EOF
+
+test_expect_success \
+ 'R: import to output marks works without any content' \
+ 'cat input | git fast-import &&
+ test_cmp marks.out marks.new'
+
+cat >input <<EOF
+feature import-marks=nonexistant.marks
+feature export-marks=marks.new
+EOF
+
+test_expect_success \
+ 'R: import marks prefers commandline marks file over the stream' \
+ 'cat input | git fast-import --import-marks=marks.out &&
+ test_cmp marks.out marks.new'
+
+
+cat >input <<EOF
+feature import-marks=nonexistant.marks
+feature export-marks=combined.marks
+EOF
+
+test_expect_success 'R: multiple --import-marks= should be honoured' '
+ head -n2 marks.out > one.marks &&
+ tail -n +3 marks.out > two.marks &&
+ git fast-import --import-marks=one.marks --import-marks=two.marks <input &&
+ test_cmp marks.out combined.marks
+'
+
+cat >input <<EOF
+feature relative-marks
+feature import-marks=relative.in
+feature export-marks=relative.out
+EOF
+
+test_expect_success 'R: feature relative-marks should be honoured' '
+ mkdir -p .git/info/fast-import/ &&
+ cp marks.new .git/info/fast-import/relative.in &&
+ git fast-import <input &&
+ test_cmp marks.new .git/info/fast-import/relative.out
+'
+
+cat >input <<EOF
+feature relative-marks
+feature import-marks=relative.in
+feature no-relative-marks
+feature export-marks=non-relative.out
+EOF
+
+test_expect_success 'R: feature no-relative-marks should be honoured' '
+ git fast-import <input &&
+ test_cmp marks.new non-relative.out
+'
+
+cat >input << EOF
+option git quiet
+blob
+data 3
+hi
+
+EOF
+
+touch empty
+
+test_expect_success 'R: quiet option results in no stats being output' '
+ cat input | git fast-import 2> output &&
+ test_cmp empty output
+'
+
+cat >input <<EOF
+option git non-existing-option
+EOF
+
+test_expect_success 'R: die on unknown option' '
+ test_must_fail git fast-import <input
+'
+
+test_expect_success 'R: unknown commandline options are rejected' '\
+ test_must_fail git fast-import --non-existing-option < /dev/null
+'
+
+cat >input <<EOF
+option non-existing-vcs non-existing-option
+EOF
+
+test_expect_success 'R: ignore non-git options' '
+ git fast-import <input
+'
+ ##
+ ## R: very large blobs
+ ##
+ blobsize=$((2*1024*1024 + 53))
+ test-genrandom bar $blobsize >expect
+ cat >input <<INPUT_END
+ commit refs/heads/big-file
+ committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+ data <<COMMIT
+ R - big file
+ COMMIT
+
+ M 644 inline big1
+ data $blobsize
+ INPUT_END
+ cat expect >>input
+ cat >>input <<INPUT_END
+ M 644 inline big2
+ data $blobsize
+ INPUT_END
+ cat expect >>input
+ echo >>input
+
+ test_expect_success \
+ 'R: blob bigger than threshold' \
+ 'test_create_repo R &&
+ git --git-dir=R/.git fast-import --big-file-threshold=1 <input'
+ test_expect_success \
+ 'R: verify created pack' \
+ ': >verify &&
+ for p in R/.git/objects/pack/*.pack;
+ do
+ git verify-pack -v $p >>verify || exit;
+ done'
+ test_expect_success \
+ 'R: verify written objects' \
+ 'git --git-dir=R/.git cat-file blob big-file:big1 >actual &&
+ test_cmp expect actual &&
+ a=$(git --git-dir=R/.git rev-parse big-file:big1) &&
+ b=$(git --git-dir=R/.git rev-parse big-file:big2) &&
+ test $a = $b'
+ test_expect_success \
+ 'R: blob appears only once' \
+ 'n=$(grep $a verify | wc -l) &&
+ test 1 = $n'
+
test_done