index-pack: loop while inflating objects in unpack_data

When the unpack_data function is given a consume() callback,
it unpacks only 64K of the input at a time, feeding it to
git_inflate along with a 64K output buffer.  However,
because we are inflating, there is a good chance that the
output buffer will fill before consuming all of the input.
In this case, we need to loop on git_inflate until we have
fed the whole input buffer, feeding each chunk of output to
the consume buffer.

The current code does not do this, and as a result, will
fail the loop condition and trigger a fatal "serious inflate
inconsistency" error in this case.

While we're rearranging the loop, let's get rid of the
extra last_out pointer. It is meant to point to the
beginning of the buffer that we feed to git_inflate, but in
practice this is always the beginning of our same 64K
buffer, because:

1. At the beginning of the loop, we are feeding the
buffer.

2. At the end of the loop, if we are using a consume()
function, we reset git_inflate's pointer to the
beginning of the buffer.  If we are not using a
consume() function, then we do not care about the value
of last_out at all.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/builtin/index-pack.c b/builtin/index-pack.c
index 8b5c1eb33e18cdec0f74e246e501f50461757e9a..50d38767b1275849bb000ec7da4c487d187ad77f 100644 (file)
--- a/builtin/index-pack.c
+++ b/builtin/index-pack.c
@@ -524,7 +524,6 @@ static void *unpack_data(struct object_entry *obj,
        stream.avail_out = consume ? 64*1024 : obj->size;
 
        do {
        stream.avail_out = consume ? 64*1024 : obj->size;
 
        do {

-               unsigned char *last_out = stream.next_out;

                ssize_t n = (len < 64*1024) ? len : 64*1024;
                n = pread(pack_fd, inbuf, n, from);
                if (n < 0)
                ssize_t n = (len < 64*1024) ? len : 64*1024;
                n = pread(pack_fd, inbuf, n, from);
                if (n < 0)


@@ -538,15 +537,19 @@ static void *unpack_data(struct object_entry *obj,
                len -= n;
                stream.next_in = inbuf;
                stream.avail_in = n;
                len -= n;
                stream.next_in = inbuf;
                stream.avail_in = n;

-               status = git_inflate(&stream, 0);
-               if (consume) {
-                       if (consume(last_out, stream.next_out - last_out, cb_data)) {
-                               free(inbuf);
-                               free(data);
-                               return NULL;
-                       }
-                       stream.next_out = data;
-                       stream.avail_out = 64*1024;
+               if (!consume)
+                       status = git_inflate(&stream, 0);
+               else {
+                       do {
+                               status = git_inflate(&stream, 0);
+                               if (consume(data, stream.next_out - data, cb_data)) {
+                                       free(inbuf);
+                                       free(data);
+                                       return NULL;
+                               }
+                               stream.next_out = data;
+                               stream.avail_out = 64*1024;
+                       } while (status == Z_OK && stream.avail_in);

                }
        } while (len && status == Z_OK && !stream.avail_in);
 
                }
        } while (len && status == Z_OK && !stream.avail_in);