X-Git-Url: https://git.tokkee.org/?a=blobdiff_plain;f=Documentation%2Fgit-fast-import.txt;h=d5119678b59492db4651006391fd693a1a12601a;hb=afc05f9f13beded8caf15d8e58d06fd64e0f7808;hp=eaba6fd4c168564f43ce2db3994dcb7b70bd4b54;hpb=070739fd351702ea9bb4f65595728ae25a488075;p=git.git diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt index eaba6fd4c..d5119678b 100644 --- a/Documentation/git-fast-import.txt +++ b/Documentation/git-fast-import.txt @@ -176,6 +176,15 @@ results, such as branch names or file names with leading or trailing spaces in their name, or early termination of fast-import when it encounters unexpected input. +Stream Comments +~~~~~~~~~~~~~~~ +To aid in debugging frontends fast-import ignores any line that +begins with `#` (ASCII pound/hash) up to and including the line +ending `LF`. A comment line may contain any sequence of bytes +that does not contain an LF and therefore may be used to include +any detailed debugging information that might be specific to the +frontend and useful when inspecting a fast-import data stream. + Date Formats ~~~~~~~~~~~~ The following date formats are supported. A frontend should select @@ -232,7 +241,7 @@ been well tested in the wild. + Frontends should prefer the `raw` format if the source material already uses UNIX-epoch format, can be coaxed to give dates in that -format, or its format is easiliy convertible to it, as there is no +format, or its format is easily convertible to it, as there is no ambiguity in parsing. `now`:: @@ -289,6 +298,11 @@ and control the current import process. More detailed discussion This command is optional and is not needed to perform an import. +`progress`:: + Causes fast-import to echo the entire line to its own + standard output. This command is optional and is not needed + to perform an import. + `commit` ~~~~~~~~ Create or update a branch with a new commit, recording one logical @@ -302,8 +316,8 @@ change to the project. data ('from' SP LF)? ('merge' SP LF)? - (filemodify | filedelete | filedeleteall)* - LF + (filemodify | filedelete | filecopy | filerename | filedeleteall)* + LF? .... where `` is the name of the branch to make the commit on. @@ -325,13 +339,17 @@ commit message use a 0 length data. Commit messages are free-form and are not interpreted by Git. Currently they must be encoded in UTF-8, as fast-import does not permit other encodings to be specified. -Zero or more `filemodify`, `filedelete` and `filedeleteall` commands +Zero or more `filemodify`, `filedelete`, `filecopy`, `filerename` +and `filedeleteall` commands may be included to update the contents of the branch prior to creating the commit. These commands may be supplied in any order. -However it is recommended that a `filedeleteall` command preceed -all `filemodify` commands in the same commit, as `filedeleteall` +However it is recommended that a `filedeleteall` command precede +all `filemodify`, `filecopy` and `filerename` commands in the same +commit, as `filedeleteall` wipes the branch clean (see below). +The `LF` after the command is optional (it used to be required). + `author` ^^^^^^^^ An `author` command may optionally appear, if the author information @@ -384,7 +402,7 @@ Here `` is any of the following: + The reason fast-import uses `:` to denote a mark reference is this character is not legal in a Git branch name. The leading `:` makes it easy -to distingush between the mark 42 (`:42`) and the branch 42 (`42` +to distinguish between the mark 42 (`:42`) and the branch 42 (`42` or `refs/heads/42`), or an abbreviated SHA-1 which happened to consist only of base-10 digits. + @@ -469,7 +487,7 @@ start with double quote (`"`). If an `LF` or double quote must be encoded into `` shell-style quoting should be used, e.g. `"path/with\n and \" in it"`. -The value of `` must be in canoncial form. That is it must not: +The value of `` must be in canonical form. That is it must not: * contain an empty directory component (e.g. `foo//bar` is invalid), * end with a directory separator (e.g. `foo/` is invalid), @@ -481,8 +499,9 @@ It is recommended that `` always be encoded using UTF-8. `filedelete` ^^^^^^^^^^^^ -Included in a `commit` command to remove a file from the branch. -If the file removal makes its directory empty, the directory will +Included in a `commit` command to remove a file or recursively +delete an entire directory from the branch. If the file or directory +removal makes its parent directory empty, the parent directory will be automatically removed too. This cascades up the tree until the first non-empty directory or the root is reached. @@ -490,9 +509,60 @@ first non-empty directory or the root is reached. 'D' SP LF .... -here `` is the complete path of the file to be removed. +here `` is the complete path of the file or subdirectory to +be removed from the branch. See `filemodify` above for a detailed description of ``. +`filecopy` +^^^^^^^^^^^^ +Recursively copies an existing file or subdirectory to a different +location within the branch. The existing file or directory must +exist. If the destination exists it will be completely replaced +by the content copied from the source. + +.... + 'C' SP SP LF +.... + +here the first `` is the source location and the second +`` is the destination. See `filemodify` above for a detailed +description of what `` may look like. To use a source path +that contains SP the path must be quoted. + +A `filecopy` command takes effect immediately. Once the source +location has been copied to the destination any future commands +applied to the source location will not impact the destination of +the copy. + +`filerename` +^^^^^^^^^^^^ +Renames an existing file or subdirectory to a different location +within the branch. The existing file or directory must exist. If +the destination exists it will be replaced by the source directory. + +.... + 'R' SP SP LF +.... + +here the first `` is the source location and the second +`` is the destination. See `filemodify` above for a detailed +description of what `` may look like. To use a source path +that contains SP the path must be quoted. + +A `filerename` command takes effect immediately. Once the source +location has been renamed to the destination any future commands +applied to the source location will create new files there and not +impact the destination of the rename. + +Note that a `filerename` is the same as a `filecopy` followed by a +`filedelete` of the source location. There is a slight performance +advantage to using `filerename`, but the advantage is so small +that it is never worth trying to convert a delete/add pair in +source material into a rename for fast-import. This `filerename` +command is provided just to simplify frontends that already have +rename information and don't want bother with decomposing it into a +`filecopy` followed by a `filedelete`. + `filedeleteall` ^^^^^^^^^^^^^^^ Included in a `commit` command to remove all files (and also all @@ -548,7 +618,6 @@ lightweight (non-annotated) tags see the `reset` command below. 'from' SP LF 'tagger' SP SP LT GT SP LF data - LF .... where `` is the name of the tag to create. @@ -592,12 +661,14 @@ branch from an existing commit without creating a new commit. .... 'reset' SP LF ('from' SP LF)? - LF + LF? .... For a detailed description of `` and `` see above under `commit` and `from`. +The `LF` after the command is optional (it used to be required). + The `reset` command can also be used to create lightweight (non-annotated) tags. For example: @@ -636,29 +707,40 @@ intended for production-quality conversions should always use the exact byte count format, as it is more robust and performs better. The delimited format is intended primarily for testing fast-import. +Comment lines appearing within the `` part of `data` commands +are always taken to be part of the body of the data and are therefore +never ignored by fast-import. This makes it safe to import any +file/message content whose lines might start with `#`. + Exact byte count format:: The frontend must specify the number of bytes of data. + .... 'data' SP LF - LF + LF? .... + where `` is the exact number of bytes appearing within ``. The value of `` is expressed as an ASCII decimal integer. The `LF` on either side of `` is not included in `` and will not be included in the imported data. ++ +The `LF` after `` is optional (it used to be required) but +recommended. Always including it makes debugging a fast-import +stream easier as the next command always starts in column 0 +of the next line, even if `` did not end with an `LF`. Delimited format:: A delimiter string is used to mark the end of the data. fast-import will compute the length by searching for the delimiter. - This format is primarly useful for testing and is not + This format is primarily useful for testing and is not recommended for real data. + .... 'data' SP '<<' LF LF LF + LF? .... + where `` is the chosen delimiter string. The string `` @@ -667,6 +749,8 @@ fast-import will think the data ends earlier than it really does. The `LF` immediately trailing `` is part of ``. This is one of the limitations of the delimited format, it is impossible to supply a data chunk which does not have an LF as its last byte. ++ +The `LF` after ` LF` is optional (it used to be required). `checkpoint` ~~~~~~~~~~~~ @@ -675,7 +759,7 @@ save out all current branch refs, tags and marks. .... 'checkpoint' LF - LF + LF? .... Note that fast-import automatically switches packfiles when the current @@ -694,6 +778,32 @@ process access to a branch. However given that a 30 GiB Subversion repository can be loaded into Git through fast-import in about 3 hours, explicit checkpointing may not be necessary. +The `LF` after the command is optional (it used to be required). + +`progress` +~~~~~~~~~~ +Causes fast-import to print the entire `progress` line unmodified to +its standard output channel (file descriptor 1) when the command is +processed from the input stream. The command otherwise has no impact +on the current import, or on any of fast-import's internal state. + +.... + 'progress' SP LF + LF? +.... + +The `` part of the command may contain any sequence of bytes +that does not contain `LF`. The `LF` after the command is optional. +Callers may wish to process the output through a tool such as sed to +remove the leading part of the line, for example: + +==== + frontend | git-fast-import | sed 's/^progress //' +==== + +Placing a `progress` command immediately after a `checkpoint` will +inform the reader when the `checkpoint` has been completed and it +can safely access the refs that fast-import updated. Tips and Tricks --------------- @@ -763,7 +873,7 @@ to remove the dummy branch. Import Now, Repack Later ~~~~~~~~~~~~~~~~~~~~~~~~ As soon as fast-import completes the Git repository is completely valid -and ready for use. Typicallly this takes only a very short time, +and ready for use. Typically this takes only a very short time, even for considerably large projects (100,000+ commits). However repacking the repository is necessary to improve data @@ -787,6 +897,15 @@ This will take longer, but will also produce a smaller packfile. You only need to expend the effort once, and everyone using your project will benefit from the smaller repository. +Include Some Progress Messages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Every once in a while have your frontend emit a `progress` message +to fast-import. The contents of the messages are entirely free-form, +so one suggestion would be to output the current month and year +each time the current commit date moves into the next month. +Your users will feel better knowing how much of the data stream +has been processed. + Packfile Optimization --------------------- @@ -823,8 +942,8 @@ Memory Utilization ------------------ There are a number of factors which affect how much memory fast-import requires to perform an import. Like critical sections of core -Git, fast-import uses its own memory allocators to ammortize any overheads -associated with malloc. In practice fast-import tends to ammoritize any +Git, fast-import uses its own memory allocators to amortize any overheads +associated with malloc. In practice fast-import tends to amortize any malloc overheads to 0, due to its use of large block allocations. per object @@ -881,7 +1000,7 @@ per active tree ~~~~~~~~~~~~~~~ Trees (aka directories) use just 12 bytes of memory on top of the memory required for their entries (see ``per active file'' below). -The cost of a tree is virtually 0, as its overhead ammortizes out +The cost of a tree is virtually 0, as its overhead amortizes out over the individual file entries. per active file entry @@ -909,4 +1028,3 @@ Documentation by Shawn O. Pearce . GIT --- Part of the gitlink:git[7] suite -