![[tokkee]](http://tokkee.org/images/avatar.png){kind=avatar}
index a92ae6d1a36f800256a477f23c8280e2717b13c9..d5119678b59492db4651006391fd693a1a12601a 100644 (file)
+
Frontends should prefer the `raw` format if the source material
already uses UNIX-epoch format, can be coaxed to give dates in that
+
Frontends should prefer the `raw` format if the source material
already uses UNIX-epoch format, can be coaxed to give dates in that
-format, or its format is easiliy convertible to it, as there is no
+format, or its format is easily convertible to it, as there is no
ambiguity in parsing.
`now`::
ambiguity in parsing.
`now`::
This command is optional and is not needed to perform
an import.
This command is optional and is not needed to perform
an import.
+`progress`::
+ Causes fast-import to echo the entire line to its own
+ standard output. This command is optional and is not needed
+ to perform an import.
+
`commit`
~~~~~~~~
Create or update a branch with a new commit, recording one logical
`commit`
~~~~~~~~
Create or update a branch with a new commit, recording one logical
('from' SP <committish> LF)?
('merge' SP <committish> LF)?
(filemodify | filedelete | filecopy | filerename | filedeleteall)*
('from' SP <committish> LF)?
('merge' SP <committish> LF)?
(filemodify | filedelete | filecopy | filerename | filedeleteall)*
- LF
+ LF?
....
where `<ref>` is the name of the branch to make the commit on.
....
where `<ref>` is the name of the branch to make the commit on.
and `filedeleteall` commands
may be included to update the contents of the branch prior to
creating the commit. These commands may be supplied in any order.
and `filedeleteall` commands
may be included to update the contents of the branch prior to
creating the commit. These commands may be supplied in any order.
-However it is recommended that a `filedeleteall` command preceed
+However it is recommended that a `filedeleteall` command precede
all `filemodify`, `filecopy` and `filerename` commands in the same
commit, as `filedeleteall`
wipes the branch clean (see below).
all `filemodify`, `filecopy` and `filerename` commands in the same
commit, as `filedeleteall`
wipes the branch clean (see below).
+The `LF` after the command is optional (it used to be required).
+
`author`
^^^^^^^^
An `author` command may optionally appear, if the author information
`author`
^^^^^^^^
An `author` command may optionally appear, if the author information
+
The reason fast-import uses `:` to denote a mark reference is this character
is not legal in a Git branch name. The leading `:` makes it easy
+
The reason fast-import uses `:` to denote a mark reference is this character
is not legal in a Git branch name. The leading `:` makes it easy
-to distingush between the mark 42 (`:42`) and the branch 42 (`42`
+to distinguish between the mark 42 (`:42`) and the branch 42 (`42`
or `refs/heads/42`), or an abbreviated SHA-1 which happened to
consist only of base-10 digits.
+
or `refs/heads/42`), or an abbreviated SHA-1 which happened to
consist only of base-10 digits.
+
If an `LF` or double quote must be encoded into `<path>` shell-style
quoting should be used, e.g. `"path/with\n and \" in it"`.
If an `LF` or double quote must be encoded into `<path>` shell-style
quoting should be used, e.g. `"path/with\n and \" in it"`.
-The value of `<path>` must be in canoncial form. That is it must not:
+The value of `<path>` must be in canonical form. That is it must not:
* contain an empty directory component (e.g. `foo//bar` is invalid),
* end with a directory separator (e.g. `foo/` is invalid),
* contain an empty directory component (e.g. `foo//bar` is invalid),
* end with a directory separator (e.g. `foo/` is invalid),
....
'reset' SP <ref> LF
('from' SP <committish> LF)?
....
'reset' SP <ref> LF
('from' SP <committish> LF)?
- LF
+ LF?
....
For a detailed description of `<ref>` and `<committish>` see above
under `commit` and `from`.
....
For a detailed description of `<ref>` and `<committish>` see above
under `commit` and `from`.
+The `LF` after the command is optional (it used to be required).
+
The `reset` command can also be used to create lightweight
(non-annotated) tags. For example:
The `reset` command can also be used to create lightweight
(non-annotated) tags. For example:
+
....
'data' SP <count> LF
+
....
'data' SP <count> LF
- <raw> LF
+ <raw> LF?
....
+
where `<count>` is the exact number of bytes appearing within
`<raw>`. The value of `<count>` is expressed as an ASCII decimal
integer. The `LF` on either side of `<raw>` is not
included in `<count>` and will not be included in the imported data.
....
+
where `<count>` is the exact number of bytes appearing within
`<raw>`. The value of `<count>` is expressed as an ASCII decimal
integer. The `LF` on either side of `<raw>` is not
included in `<count>` and will not be included in the imported data.
++
+The `LF` after `<raw>` is optional (it used to be required) but
+recommended. Always including it makes debugging a fast-import
+stream easier as the next command always starts in column 0
+of the next line, even if `<raw>` did not end with an `LF`.
Delimited format::
A delimiter string is used to mark the end of the data.
fast-import will compute the length by searching for the delimiter.
Delimited format::
A delimiter string is used to mark the end of the data.
fast-import will compute the length by searching for the delimiter.
- This format is primarly useful for testing and is not
+ This format is primarily useful for testing and is not
recommended for real data.
+
....
'data' SP '<<' <delim> LF
<raw> LF
<delim> LF
recommended for real data.
+
....
'data' SP '<<' <delim> LF
<raw> LF
<delim> LF
+ LF?
....
+
where `<delim>` is the chosen delimiter string. The string `<delim>`
....
+
where `<delim>` is the chosen delimiter string. The string `<delim>`
immediately trailing `<raw>` is part of `<raw>`. This is one of
the limitations of the delimited format, it is impossible to supply
a data chunk which does not have an LF as its last byte.
immediately trailing `<raw>` is part of `<raw>`. This is one of
the limitations of the delimited format, it is impossible to supply
a data chunk which does not have an LF as its last byte.
++
+The `LF` after `<delim> LF` is optional (it used to be required).
`checkpoint`
~~~~~~~~~~~~
`checkpoint`
~~~~~~~~~~~~
....
'checkpoint' LF
....
'checkpoint' LF
- LF
+ LF?
....
Note that fast-import automatically switches packfiles when the current
....
Note that fast-import automatically switches packfiles when the current
repository can be loaded into Git through fast-import in about 3 hours,
explicit checkpointing may not be necessary.
repository can be loaded into Git through fast-import in about 3 hours,
explicit checkpointing may not be necessary.
+The `LF` after the command is optional (it used to be required).
+
+`progress`
+~~~~~~~~~~
+Causes fast-import to print the entire `progress` line unmodified to
+its standard output channel (file descriptor 1) when the command is
+processed from the input stream. The command otherwise has no impact
+on the current import, or on any of fast-import's internal state.
+
+....
+ 'progress' SP <any> LF
+ LF?
+....
+
+The `<any>` part of the command may contain any sequence of bytes
+that does not contain `LF`. The `LF` after the command is optional.
+Callers may wish to process the output through a tool such as sed to
+remove the leading part of the line, for example:
+
+====
+ frontend | git-fast-import | sed 's/^progress //'
+====
+
+Placing a `progress` command immediately after a `checkpoint` will
+inform the reader when the `checkpoint` has been completed and it
+can safely access the refs that fast-import updated.
Tips and Tricks
---------------
Tips and Tricks
---------------
Import Now, Repack Later
~~~~~~~~~~~~~~~~~~~~~~~~
As soon as fast-import completes the Git repository is completely valid
Import Now, Repack Later
~~~~~~~~~~~~~~~~~~~~~~~~
As soon as fast-import completes the Git repository is completely valid
-and ready for use. Typicallly this takes only a very short time,
+and ready for use. Typically this takes only a very short time,
even for considerably large projects (100,000+ commits).
However repacking the repository is necessary to improve data
even for considerably large projects (100,000+ commits).
However repacking the repository is necessary to improve data
You only need to expend the effort once, and everyone using your
project will benefit from the smaller repository.
You only need to expend the effort once, and everyone using your
project will benefit from the smaller repository.
+Include Some Progress Messages
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Every once in a while have your frontend emit a `progress` message
+to fast-import. The contents of the messages are entirely free-form,
+so one suggestion would be to output the current month and year
+each time the current commit date moves into the next month.
+Your users will feel better knowing how much of the data stream
+has been processed.
+
Packfile Optimization
---------------------
Packfile Optimization
---------------------
------------------
There are a number of factors which affect how much memory fast-import
requires to perform an import. Like critical sections of core
------------------
There are a number of factors which affect how much memory fast-import
requires to perform an import. Like critical sections of core
-Git, fast-import uses its own memory allocators to ammortize any overheads
-associated with malloc. In practice fast-import tends to ammoritize any
+Git, fast-import uses its own memory allocators to amortize any overheads
+associated with malloc. In practice fast-import tends to amortize any
malloc overheads to 0, due to its use of large block allocations.
per object
malloc overheads to 0, due to its use of large block allocations.
per object
~~~~~~~~~~~~~~~
Trees (aka directories) use just 12 bytes of memory on top of the
memory required for their entries (see ``per active file'' below).
~~~~~~~~~~~~~~~
Trees (aka directories) use just 12 bytes of memory on top of the
memory required for their entries (see ``per active file'' below).
-The cost of a tree is virtually 0, as its overhead ammortizes out
+The cost of a tree is virtually 0, as its overhead amortizes out
over the individual file entries.
per active file entry
over the individual file entries.
per active file entry