Dump all refs and marks during a checkpoint in fast-import.

If the frontend asks us to checkpoint (via the explicit checkpoint
command) its probably because they are afraid the current import
will crash/fail/whatever and want to make sure they can pickup from
the last checkpoint.  To do that sort of recovery, we will need the
current tip of every branch and tag available at the next startup.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>

diff --git a/Documentation/git-fast-import.txt b/Documentation/git-fast-import.txt
index beab7f0a54810824a9a6737a819b025f66d32933..1e12d210c9347e810bf0e04a41819e57f990f4e1 100644 (file)
--- a/Documentation/git-fast-import.txt
+++ b/Documentation/git-fast-import.txt
@@ -650,17 +650,31 @@ a data chunk which does not have an LF as its last byte.
 
 `checkpoint`
 ~~~~~~~~~~~~
 
 `checkpoint`
 ~~~~~~~~~~~~

-Forces gfi to close the current packfile and start a new one.
-As this requires a significant amount of CPU time and disk IO
-(to compute the overall pack SHA-1 checksum and generate the
-corresponding index file) it can easily take several minutes for
-a single `checkpoint` command to complete.
+Forces gfi to close the current packfile, start a new one, and to
+save out all current branch refs, tags and marks.

 
 ....
        'checkpoint' LF
        LF
 ....
 
 
 ....
        'checkpoint' LF
        LF
 ....
 

+Note that gfi automatically switches packfiles when the current
+packfile reaches \--max-pack-size, or 4 GiB, whichever limit is
+smaller.  During an automatic packfile switch gfi does not update
+the branch refs, tags or marks.
+
+As a `checkpoint` can require a significant amount of CPU time and
+disk IO (to compute the overall pack SHA-1 checksum, generate the
+corresponding index file, and update the refs) it can easily take
+several minutes for a single `checkpoint` command to complete.
+
+Frontends may choose to issue checkpoints during extremely large
+and long running imports, or when they need to allow another Git
+process access to a branch.  However given that a 30 GiB Subversion
+repository can be loaded into Git through gfi in about 3 hours,
+explicit checkpointing may not be necessary.
+
+

 Packfile Optimization
 ---------------------
 When packing a blob gfi always attempts to deltify against the last
 Packfile Optimization
 ---------------------
 When packing a blob gfi always attempts to deltify against the last

diff --git a/fast-import.c b/fast-import.c
index 858df17f35efe2ab610c501744ec65b621a8f53d..d9ed3e2f18ccba41da216c9a94031b2ae44025c8 100644 (file)
--- a/fast-import.c
+++ b/fast-import.c
@@ -838,7 +838,7 @@ static void end_packfile(void)
        last_blob.depth = 0;
 }
 
        last_blob.depth = 0;
 }
 

-static void checkpoint(void)
+static void cycle_packfile(void)

 {
        end_packfile();
        start_packfile();
 {
        end_packfile();
        start_packfile();


@@ -931,7 +931,7 @@ static int store_object(
 
                /* This new object needs to *not* have the current pack_id. */
                e->pack_id = pack_id + 1;
 
                /* This new object needs to *not* have the current pack_id. */
                e->pack_id = pack_id + 1;

-               checkpoint();
+               cycle_packfile();

 
                /* We cannot carry a delta into the new pack. */
                if (delta) {
 
                /* We cannot carry a delta into the new pack. */
                if (delta) {


@@ -1940,8 +1940,12 @@ static void cmd_reset_branch(void)
 
 static void cmd_checkpoint(void)
 {
 
 static void cmd_checkpoint(void)
 {

-       if (object_count)
-               checkpoint();
+       if (object_count) {
+               cycle_packfile();
+               dump_branches();
+               dump_tags();
+               dump_marks();
+       }

        read_next_command();
 }
 
        read_next_command();
 }