From: Junio C Hamano <junkio@cox.net>
Date: Mon, 8 May 2006 23:42:12 +0000 (-0700)
Subject: Merge branch 'jc/bindiff' into next
X-Git-Tag: v1.4.1-rc1~122
X-Git-Url: https://git.lorimer.id.au/gitweb.git/diff_plain/dd1139a94c0250640d4fceecea0714a8420e920b?hp=addaacab07d91e5ff1f06ada31c9e96c0edd31cd

Merge branch 'jc/bindiff' into next

* jc/bindiff:
improve base85 generated assembly code
---

diff --git a/.gitignore b/.gitignore
index b5959d6311..7906909b30 100644
--- a/.gitignore
+++ b/.gitignore
@@ -123,6 +123,7 @@ git-write-tree
 git-core-*/?*
 test-date
 test-delta
+test-dump-cache-tree
 common-cmds.h
 *.tar.gz
 *.dsc
diff --git a/Documentation/core-tutorial.txt b/Documentation/core-tutorial.txt
index 4211c81972..d1360ecde2 100644
--- a/Documentation/core-tutorial.txt
+++ b/Documentation/core-tutorial.txt
@@ -971,7 +971,7 @@ $ git show-branch --topo-order master mybranch
 The first two lines indicate that it is showing the two branches
 and the first line of the commit log message from their
 top-of-the-tree commits, you are currently on `master` branch
-(notice the asterisk `*` character), and the first column for
+(notice the asterisk `\*` character), and the first column for
 the later output lines is used to show commits contained in the
 `master` branch, and the second column for the `mybranch`
 branch. Three commits are shown along with their log messages.
diff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt
index d55456ae93..56b683ae56 100644
--- a/Documentation/git-grep.txt
+++ b/Documentation/git-grep.txt
@@ -8,43 +8,82 @@ git-grep - Print lines matching a pattern
 
 SYNOPSIS
 --------
-'git-grep' [<option>...] [-e] <pattern> [--] [<path>...]
+[verse]
+'git-grep' [--cached]
+	   [-a | --text] [-I] [-i | --ignore-case] [-w | --word-regexp]
+	   [-v | --invert-match]
+	   [-E | --extended-regexp]  [-G | --basic-regexp]
+	   [-n] [-l | --files-with-matches] [-L | --files-without-match]
+	   [-c | --count]
+	   [-A <post-context>] [-B <pre-context>] [-C <context>]
+	   [-f <file>] [-e <pattern>]
+	   [<tree>...]
+	   [--] [<path>...]
 
 DESCRIPTION
 -----------
-Searches list of files `git-ls-files` produces for lines
-containing a match to the given pattern.
+Look for specified patterns in the working tree files, blobs
+registered in the index file, or given tree objects.
 
 
 OPTIONS
 -------
-`--`::
-	Signals the end of options; the rest of the parameters
-	are <path> limiters.
+--cached::
+	Instead of searching in the working tree files, check
+	the blobs registerd in the index file.
+
+-a | --text::
+	Process binary files as if they were text.
+
+-i | --ignore-case::
+	Ignore case differences between the patterns and the
+	files.
+
+-w | --word-regexp::
+	Match the pattern only at word boundary (either begin at the
+	beginning of a line, or preceded by a non-word character; end at
+	the end of a line or followed by a non-word character).
+
+-v | --invert-match::
+	Select non-matching lines.
+
+-E | --extended-regexp | -G | --basic-regexp::
+	Use POSIX extended/basic regexp for patterns.  Default
+	is to use basic regexp.
 
-<option>...::
-	Either an option to pass to `grep` or `git-ls-files`.
-+
-The following are the specific `git-ls-files` options
-that may be given: `-o`, `--cached`, `--deleted`, `--others`,
-`--killed`, `--ignored`, `--modified`, `--exclude=\*`,
-`--exclude-from=\*`, and `--exclude-per-directory=\*`.
-+
-All other options will be passed to `grep`.
+-n::
+	Prefix the line number to matching lines.
 
-<pattern>::
-	The pattern to look for.  The first non option is taken
-	as the pattern; if your pattern begins with a dash, use
-	`-e <pattern>`.
+-l | --files-with-matches | -L | --files-without-match::
+	Instead of showing every matched line, show only the
+	names of files that contain (or do not contain) matches.
 
-<path>...::
-	Optional paths to limit the set of files to be searched;
-	passed to `git-ls-files`.
+-c | --count::
+	Instead of showing every matched line, show the number of
+	lines that match.
+
+-[ABC] <context>::
+	Show `context` trailing (`A` -- after), or leading (`B`
+	-- before), or both (`C` -- context) lines, and place a
+	line containing `--` between continguous groups of
+	matches.
+
+-f <file>::
+	Read patterns from <file>, one per line.
+
+`<tree>...`::
+	Search blobs in the trees for specified patterns.
+
+`--`::
+	Signals the end of options; the rest of the parameters
+	are <path> limiters.
 
 
 Author
 ------
-Written by Linus Torvalds <torvalds@osdl.org>
+Originally written by Linus Torvalds <torvalds@osdl.org>, later
+revamped by Junio C Hamano.
+
 
 Documentation
 --------------
diff --git a/Documentation/git-read-tree.txt b/Documentation/git-read-tree.txt
index 844cfda8d2..1f21d95684 100644
--- a/Documentation/git-read-tree.txt
+++ b/Documentation/git-read-tree.txt
@@ -8,7 +8,7 @@ git-read-tree - Reads tree information into the index
 
 SYNOPSIS
 --------
-'git-read-tree' (<tree-ish> | [[-m [--aggressive]| --reset] [-u | -i]] <tree-ish1> [<tree-ish2> [<tree-ish3>]])
+'git-read-tree' (<tree-ish> | [[-m [--aggressive] | --reset | --prefix=<prefix>] [-u | -i]] <tree-ish1> [<tree-ish2> [<tree-ish3>]])
 
 
 DESCRIPTION
@@ -63,6 +63,15 @@ OPTIONS
 * when both sides adds a path identically.  The resolution
   is to add that path.
 
+--prefix=<prefix>/::
+	Keep the current index contents, and read the contents
+	of named tree-ish under directory at `<prefix>`.  The
+	original index file cannot have anything at the path
+	`<prefix>` itself, and have nothing in `<prefix>/`
+	directory.  Note that the `<prefix>/` value must end
+	with a slash.
+
+
 <tree-ish#>::
 	The id of the tree object(s) to be read/merged.
 
diff --git a/Documentation/git-repo-config.txt b/Documentation/git-repo-config.txt
index fd44f629f5..660c18ff8d 100644
--- a/Documentation/git-repo-config.txt
+++ b/Documentation/git-repo-config.txt
@@ -23,10 +23,11 @@ You can query/set/replace/unset options with this command. The name is
 actually the section and the key separated by a dot, and the value will be
 escaped.
 
-If you want to set/unset an option which can occur on multiple lines, you
-should provide a POSIX regex for the value. If you want to handle the lines
-*not* matching the regex, just prepend a single exclamation mark in front
-(see EXAMPLES).
+If you want to set/unset an option which can occur on multiple
+lines, a POSIX regexp `value_regex` needs to be given.  Only the
+existing values that match the regexp are updated or unset.  If
+you want to handle the lines that do *not* match the regex, just
+prepend a single exclamation mark in front (see EXAMPLES).
 
 The type specifier can be either '--int' or '--bool', which will make
 'git-repo-config' ensure that the variable(s) are of the given type and
diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt
index 57177c7fb6..d043e86a77 100644
--- a/Documentation/git-update-index.txt
+++ b/Documentation/git-update-index.txt
@@ -15,7 +15,7 @@ SYNOPSIS
 	     [--cacheinfo <mode> <object> <file>]\*
 	     [--chmod=(+|-)x]
 	     [--assume-unchanged | --no-assume-unchanged]
-	     [--really-refresh] [--unresolve]
+	     [--really-refresh] [--unresolve] [--again]
 	     [--info-only] [--index-info]
 	     [-z] [--stdin]
 	     [--verbose]
@@ -80,6 +80,10 @@ OPTIONS
 	filesystem that has very slow lstat(2) system call
 	(e.g. cifs).
 
+--again::
+	Runs `git-update-index` itself on the paths whose index
+	entries are different from those from the `HEAD` commit.
+
 --unresolve::
 	Restores the 'unmerged' or 'needs updating' state of a
 	file during a merge if it was cleared by accident.
diff --git a/Documentation/git-write-tree.txt b/Documentation/git-write-tree.txt
index 77e12cb949..c85fa89c30 100644
--- a/Documentation/git-write-tree.txt
+++ b/Documentation/git-write-tree.txt
@@ -8,7 +8,7 @@ git-write-tree - Creates a tree object from the current index
 
 SYNOPSIS
 --------
-'git-write-tree' [--missing-ok]
+'git-write-tree' [--missing-ok] [--prefix=<prefix>/]
 
 DESCRIPTION
 -----------
@@ -30,6 +30,12 @@ OPTIONS
 	directory exist in the object database.  This option disables this
 	check.
 
+--prefix=<prefix>/::
+	Writes a tree object that represents a subdirectory
+	`<prefix>`.  This can be used to write the tree object
+	for a subproject that is in the named subdirectory.
+
+
 Author
 ------
 Written by Linus Torvalds <torvalds@osdl.org>
diff --git a/Makefile b/Makefile
index 814010d7b4..070c478bf2 100644
--- a/Makefile
+++ b/Makefile
@@ -205,7 +205,7 @@ DIFF_OBJS = \
 	diffcore-delta.o log-tree.o
 
 LIB_OBJS = \
-	blob.o commit.o connect.o csum-file.o base85.o \
+	blob.o commit.o connect.o csum-file.o cache-tree.o base85.o \
 	date.o diff-delta.o entry.o exec_cmd.o ident.o index.o \
 	object.o pack-check.o patch-delta.o path.o pkt-line.o \
 	quote.o read-cache.o refs.o run-command.o \
@@ -215,7 +215,8 @@ LIB_OBJS = \
 	$(DIFF_OBJS)
 
 BUILTIN_OBJS = \
-	builtin-log.o builtin-help.o builtin-count.o builtin-diff.o builtin-push.o
+	builtin-log.o builtin-help.o builtin-count.o builtin-diff.o builtin-push.o \
+	builtin-grep.o
 
 GITLIBS = $(LIB_FILE) $(XDIFF_LIB)
 LIBS = $(GITLIBS) -lz
@@ -607,7 +608,10 @@ test-date$X: test-date.c date.o ctype.o
 	$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) test-date.c date.o ctype.o
 
 test-delta$X: test-delta.c diff-delta.o patch-delta.o
-	$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $^ -lz
+	$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $^
+
+test-dump-cache-tree$X: dump-cache-tree.o $(GITLIBS)
+	$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(LIBS)
 
 check:
 	for i in *.c; do sparse $(ALL_CFLAGS) $(SPARSE_FLAGS) $$i || exit; done
diff --git a/apply.c b/apply.c
index 1b93aab8af..88d2a3206b 100644
--- a/apply.c
+++ b/apply.c
@@ -8,6 +8,7 @@
  */
 #include <fnmatch.h>
 #include "cache.h"
+#include "cache-tree.h"
 #include "quote.h"
 #include "blob.h"
 #include "delta.h"
@@ -1892,6 +1893,7 @@ static void remove_file(struct patch *patch)
 	if (write_index) {
 		if (remove_file_from_cache(patch->old_name) < 0)
 			die("unable to remove %s from index", patch->old_name);
+		cache_tree_invalidate_path(active_cache_tree, patch->old_name);
 	}
 	unlink(patch->old_name);
 }
@@ -1988,8 +1990,9 @@ static void create_file(struct patch *patch)
 
 	if (!mode)
 		mode = S_IFREG | 0644;
-	create_one_file(path, mode, buf, size);	
+	create_one_file(path, mode, buf, size);
 	add_index_file(path, mode, buf, size);
+	cache_tree_invalidate_path(active_cache_tree, path);
 }
 
 static void write_out_one_result(struct patch *patch)
diff --git a/builtin-diff.c b/builtin-diff.c
index 636edbf2a7..71742aa10b 100644
--- a/builtin-diff.c
+++ b/builtin-diff.c
@@ -84,8 +84,7 @@ static void stuff_change(struct diff_options *opt,
 
 	if (opt->reverse_diff) {
 		unsigned tmp;
-		const
-			const unsigned char *tmp_u;
+		const unsigned char *tmp_u;
 		const char *tmp_c;
 		tmp = old_mode; old_mode = new_mode; new_mode = tmp;
 		tmp_u = old_sha1; old_sha1 = new_sha1; new_sha1 = tmp_u;
@@ -232,7 +231,7 @@ static int builtin_diff_combined(struct rev_info *revs,
 	return 0;
 }
 
-static void add_head(struct rev_info *revs)
+void add_head(struct rev_info *revs)
 {
 	unsigned char sha1[20];
 	struct object *obj;
diff --git a/builtin-grep.c b/builtin-grep.c
new file mode 100644
index 0000000000..a762c48a5a
--- /dev/null
+++ b/builtin-grep.c
@@ -0,0 +1,705 @@
+/*
+ * Builtin "git grep"
+ *
+ * Copyright (c) 2006 Junio C Hamano
+ */
+#include "cache.h"
+#include "blob.h"
+#include "tree.h"
+#include "commit.h"
+#include "tag.h"
+#include "tree-walk.h"
+#include "builtin.h"
+#include <regex.h>
+#include <fnmatch.h>
+
+/*
+ * git grep pathspecs are somewhat different from diff-tree pathspecs;
+ * pathname wildcards are allowed.
+ */
+static int pathspec_matches(const char **paths, const char *name)
+{
+	int namelen, i;
+	if (!paths || !*paths)
+		return 1;
+	namelen = strlen(name);
+	for (i = 0; paths[i]; i++) {
+		const char *match = paths[i];
+		int matchlen = strlen(match);
+		const char *cp, *meta;
+
+		if ((matchlen <= namelen) &&
+		    !strncmp(name, match, matchlen) &&
+		    (match[matchlen-1] == '/' ||
+		     name[matchlen] == '\0' || name[matchlen] == '/'))
+			return 1;
+		if (!fnmatch(match, name, 0))
+			return 1;
+		if (name[namelen-1] != '/')
+			continue;
+
+		/* We are being asked if the directory ("name") is worth
+		 * descending into.
+		 *
+		 * Find the longest leading directory name that does
+		 * not have metacharacter in the pathspec; the name
+		 * we are looking at must overlap with that directory.
+		 */
+		for (cp = match, meta = NULL; cp - match < matchlen; cp++) {
+			char ch = *cp;
+			if (ch == '*' || ch == '[' || ch == '?') {
+				meta = cp;
+				break;
+			}
+		}
+		if (!meta)
+			meta = cp; /* fully literal */
+
+		if (namelen <= meta - match) {
+			/* Looking at "Documentation/" and
+			 * the pattern says "Documentation/howto/", or
+			 * "Documentation/diff*.txt".  The name we
+			 * have should match prefix.
+			 */
+			if (!memcmp(match, name, namelen))
+				return 1;
+			continue;
+		}
+
+		if (meta - match < namelen) {
+			/* Looking at "Documentation/howto/" and
+			 * the pattern says "Documentation/h*";
+			 * match up to "Do.../h"; this avoids descending
+			 * into "Documentation/technical/".
+			 */
+			if (!memcmp(match, name, meta - match))
+				return 1;
+			continue;
+		}
+	}
+	return 0;
+}
+
+struct grep_pat {
+	struct grep_pat *next;
+	const char *origin;
+	int no;
+	const char *pattern;
+	regex_t regexp;
+};
+
+struct grep_opt {
+	struct grep_pat *pattern_list;
+	struct grep_pat **pattern_tail;
+	regex_t regexp;
+	unsigned linenum:1;
+	unsigned invert:1;
+	unsigned name_only:1;
+	unsigned unmatch_name_only:1;
+	unsigned count:1;
+	unsigned word_regexp:1;
+#define GREP_BINARY_DEFAULT	0
+#define GREP_BINARY_NOMATCH	1
+#define GREP_BINARY_TEXT	2
+	unsigned binary:2;
+	int regflags;
+	unsigned pre_context;
+	unsigned post_context;
+};
+
+static void add_pattern(struct grep_opt *opt, const char *pat,
+			const char *origin, int no)
+{
+	struct grep_pat *p = xcalloc(1, sizeof(*p));
+	p->pattern = pat;
+	p->origin = origin;
+	p->no = no;
+	*opt->pattern_tail = p;
+	opt->pattern_tail = &p->next;
+	p->next = NULL;
+}
+
+static void compile_patterns(struct grep_opt *opt)
+{
+	struct grep_pat *p;
+	for (p = opt->pattern_list; p; p = p->next) {
+		int err = regcomp(&p->regexp, p->pattern, opt->regflags);
+		if (err) {
+			char errbuf[1024];
+			char where[1024];
+			if (p->no)
+				sprintf(where, "In '%s' at %d, ",
+					p->origin, p->no);
+			else if (p->origin)
+				sprintf(where, "%s, ", p->origin);
+			else
+				where[0] = 0;
+			regerror(err, &p->regexp, errbuf, 1024);
+			regfree(&p->regexp);
+			die("%s'%s': %s", where, p->pattern, errbuf);
+		}
+	}
+}
+
+static char *end_of_line(char *cp, unsigned long *left)
+{
+	unsigned long l = *left;
+	while (l && *cp != '\n') {
+		l--;
+		cp++;
+	}
+	*left = l;
+	return cp;
+}
+
+static int word_char(char ch)
+{
+	return isalnum(ch) || ch == '_';
+}
+
+static void show_line(struct grep_opt *opt, const char *bol, const char *eol,
+		      const char *name, unsigned lno, char sign)
+{
+	printf("%s%c", name, sign);
+	if (opt->linenum)
+		printf("%d%c", lno, sign);
+	printf("%.*s\n", (int)(eol-bol), bol);
+}
+
+/*
+ * NEEDSWORK: share code with diff.c
+ */
+#define FIRST_FEW_BYTES 8000
+static int buffer_is_binary(const char *ptr, unsigned long size)
+{
+	if (FIRST_FEW_BYTES < size)
+		size = FIRST_FEW_BYTES;
+	if (memchr(ptr, 0, size))
+		return 1;
+	return 0;
+}
+
+static int grep_buffer(struct grep_opt *opt, const char *name,
+		       char *buf, unsigned long size)
+{
+	char *bol = buf;
+	unsigned long left = size;
+	unsigned lno = 1;
+	struct pre_context_line {
+		char *bol;
+		char *eol;
+	} *prev = NULL, *pcl;
+	unsigned last_hit = 0;
+	unsigned last_shown = 0;
+	int binary_match_only = 0;
+	const char *hunk_mark = "";
+	unsigned count = 0;
+
+	if (buffer_is_binary(buf, size)) {
+		switch (opt->binary) {
+		case GREP_BINARY_DEFAULT:
+			binary_match_only = 1;
+			break;
+		case GREP_BINARY_NOMATCH:
+			return 0; /* Assume unmatch */
+			break;
+		default:
+			break;
+		}
+	}
+
+	if (opt->pre_context)
+		prev = xcalloc(opt->pre_context, sizeof(*prev));
+	if (opt->pre_context || opt->post_context)
+		hunk_mark = "--\n";
+
+	while (left) {
+		regmatch_t pmatch[10];
+		char *eol, ch;
+		int hit = 0;
+		struct grep_pat *p;
+
+		eol = end_of_line(bol, &left);
+		ch = *eol;
+		*eol = 0;
+
+		for (p = opt->pattern_list; p; p = p->next) {
+			regex_t *exp = &p->regexp;
+			hit = !regexec(exp, bol, ARRAY_SIZE(pmatch),
+				       pmatch, 0);
+
+			if (hit && opt->word_regexp) {
+				/* Match beginning must be either
+				 * beginning of the line, or at word
+				 * boundary (i.e. the last char must
+				 * not be alnum or underscore).
+				 */
+				if ((pmatch[0].rm_so < 0) ||
+				    (eol - bol) <= pmatch[0].rm_so ||
+				    (pmatch[0].rm_eo < 0) ||
+				    (eol - bol) < pmatch[0].rm_eo)
+					die("regexp returned nonsense");
+				if (pmatch[0].rm_so != 0 &&
+				    word_char(bol[pmatch[0].rm_so-1]))
+					continue; /* not a word boundary */
+				if ((eol-bol) < pmatch[0].rm_eo &&
+				    word_char(bol[pmatch[0].rm_eo]))
+					continue; /* not a word boundary */
+			}
+			if (hit)
+				break;
+		}
+		/* "grep -v -e foo -e bla" should list lines
+		 * that do not have either, so inversion should
+		 * be done outside.
+		 */
+		if (opt->invert)
+			hit = !hit;
+		if (opt->unmatch_name_only) {
+			if (hit)
+				return 0;
+			goto next_line;
+		}
+		if (hit) {
+			count++;
+			if (binary_match_only) {
+				printf("Binary file %s matches\n", name);
+				return 1;
+			}
+			if (opt->name_only) {
+				printf("%s\n", name);
+				return 1;
+			}
+			/* Hit at this line.  If we haven't shown the
+			 * pre-context lines, we would need to show them.
+			 * When asked to do "count", this still show
+			 * the context which is nonsense, but the user
+			 * deserves to get that ;-).
+			 */
+			if (opt->pre_context) {
+				unsigned from;
+				if (opt->pre_context < lno)
+					from = lno - opt->pre_context;
+				else
+					from = 1;
+				if (from <= last_shown)
+					from = last_shown + 1;
+				if (last_shown && from != last_shown + 1)
+					printf(hunk_mark);
+				while (from < lno) {
+					pcl = &prev[lno-from-1];
+					show_line(opt, pcl->bol, pcl->eol,
+						  name, from, '-');
+					from++;
+				}
+				last_shown = lno-1;
+			}
+			if (last_shown && lno != last_shown + 1)
+				printf(hunk_mark);
+			if (!opt->count)
+				show_line(opt, bol, eol, name, lno, ':');
+			last_shown = last_hit = lno;
+		}
+		else if (last_hit &&
+			 lno <= last_hit + opt->post_context) {
+			/* If the last hit is within the post context,
+			 * we need to show this line.
+			 */
+			if (last_shown && lno != last_shown + 1)
+				printf(hunk_mark);
+			show_line(opt, bol, eol, name, lno, '-');
+			last_shown = lno;
+		}
+		if (opt->pre_context) {
+			memmove(prev+1, prev,
+				(opt->pre_context-1) * sizeof(*prev));
+			prev->bol = bol;
+			prev->eol = eol;
+		}
+
+	next_line:
+		*eol = ch;
+		bol = eol + 1;
+		if (!left)
+			break;
+		left--;
+		lno++;
+	}
+
+	if (opt->unmatch_name_only) {
+		/* We did not see any hit, so we want to show this */
+		printf("%s\n", name);
+		return 1;
+	}
+
+	/* NEEDSWORK:
+	 * The real "grep -c foo *.c" gives many "bar.c:0" lines,
+	 * which feels mostly useless but sometimes useful.  Maybe
+	 * make it another option?  For now suppress them.
+	 */
+	if (opt->count && count)
+		printf("%s:%u\n", name, count);
+	return !!last_hit;
+}
+
+static int grep_sha1(struct grep_opt *opt, const unsigned char *sha1, const char *name)
+{
+	unsigned long size;
+	char *data;
+	char type[20];
+	int hit;
+	data = read_sha1_file(sha1, type, &size);
+	if (!data) {
+		error("'%s': unable to read %s", name, sha1_to_hex(sha1));
+		return 0;
+	}
+	hit = grep_buffer(opt, name, data, size);
+	free(data);
+	return hit;
+}
+
+static int grep_file(struct grep_opt *opt, const char *filename)
+{
+	struct stat st;
+	int i;
+	char *data;
+	if (lstat(filename, &st) < 0) {
+	err_ret:
+		if (errno != ENOENT)
+			error("'%s': %s", filename, strerror(errno));
+		return 0;
+	}
+	if (!st.st_size)
+		return 0; /* empty file -- no grep hit */
+	if (!S_ISREG(st.st_mode))
+		return 0;
+	i = open(filename, O_RDONLY);
+	if (i < 0)
+		goto err_ret;
+	data = xmalloc(st.st_size + 1);
+	if (st.st_size != xread(i, data, st.st_size)) {
+		error("'%s': short read %s", filename, strerror(errno));
+		close(i);
+		free(data);
+		return 0;
+	}
+	close(i);
+	i = grep_buffer(opt, filename, data, st.st_size);
+	free(data);
+	return i;
+}
+
+static int grep_cache(struct grep_opt *opt, const char **paths, int cached)
+{
+	int hit = 0;
+	int nr;
+	read_cache();
+
+	for (nr = 0; nr < active_nr; nr++) {
+		struct cache_entry *ce = active_cache[nr];
+		if (ce_stage(ce) || !S_ISREG(ntohl(ce->ce_mode)))
+			continue;
+		if (!pathspec_matches(paths, ce->name))
+			continue;
+		if (cached)
+			hit |= grep_sha1(opt, ce->sha1, ce->name);
+		else
+			hit |= grep_file(opt, ce->name);
+	}
+	return hit;
+}
+
+static int grep_tree(struct grep_opt *opt, const char **paths,
+		     struct tree_desc *tree,
+		     const char *tree_name, const char *base)
+{
+	unsigned mode;
+	int len;
+	int hit = 0;
+	const char *path;
+	const unsigned char *sha1;
+	char *down;
+	char *path_buf = xmalloc(PATH_MAX + strlen(tree_name) + 100);
+
+	if (tree_name[0]) {
+		int offset = sprintf(path_buf, "%s:", tree_name);
+		down = path_buf + offset;
+		strcat(down, base);
+	}
+	else {
+		down = path_buf;
+		strcpy(down, base);
+	}
+	len = strlen(path_buf);
+
+	while (tree->size) {
+		int pathlen;
+		sha1 = tree_entry_extract(tree, &path, &mode);
+		pathlen = strlen(path);
+		strcpy(path_buf + len, path);
+
+		if (S_ISDIR(mode))
+			/* Match "abc/" against pathspec to
+			 * decide if we want to descend into "abc"
+			 * directory.
+			 */
+			strcpy(path_buf + len + pathlen, "/");
+
+		if (!pathspec_matches(paths, down))
+			;
+		else if (S_ISREG(mode))
+			hit |= grep_sha1(opt, sha1, path_buf);
+		else if (S_ISDIR(mode)) {
+			char type[20];
+			struct tree_desc sub;
+			void *data;
+			data = read_sha1_file(sha1, type, &sub.size);
+			if (!data)
+				die("unable to read tree (%s)",
+				    sha1_to_hex(sha1));
+			sub.buf = data;
+			hit |= grep_tree(opt, paths, &sub, tree_name, down);
+			free(data);
+		}
+		update_tree_entry(tree);
+	}
+	return hit;
+}
+
+static int grep_object(struct grep_opt *opt, const char **paths,
+		       struct object *obj, const char *name)
+{
+	if (!strcmp(obj->type, blob_type))
+		return grep_sha1(opt, obj->sha1, name);
+	if (!strcmp(obj->type, commit_type) ||
+	    !strcmp(obj->type, tree_type)) {
+		struct tree_desc tree;
+		void *data;
+		int hit;
+		data = read_object_with_reference(obj->sha1, tree_type,
+						  &tree.size, NULL);
+		if (!data)
+			die("unable to read tree (%s)", sha1_to_hex(obj->sha1));
+		tree.buf = data;
+		hit = grep_tree(opt, paths, &tree, name, "");
+		free(data);
+		return hit;
+	}
+	die("unable to grep from object of type %s", obj->type);
+}
+
+static const char builtin_grep_usage[] =
+"git-grep <option>* <rev>* [-e] <pattern> [<path>...]";
+
+int cmd_grep(int argc, const char **argv, char **envp)
+{
+	int hit = 0;
+	int no_more_flags = 0;
+	int cached = 0;
+	struct grep_opt opt;
+	struct object_list *list, **tail, *object_list = NULL;
+	const char *prefix = setup_git_directory();
+	const char **paths = NULL;
+
+	memset(&opt, 0, sizeof(opt));
+	opt.pattern_tail = &opt.pattern_list;
+	opt.regflags = REG_NEWLINE;
+
+	/*
+	 * No point using rev_info, really.
+	 */
+	while (1 < argc) {
+		const char *arg = argv[1];
+		argc--; argv++;
+		if (!strcmp("--cached", arg)) {
+			cached = 1;
+			continue;
+		}
+		if (!strcmp("-a", arg) ||
+		    !strcmp("--text", arg)) {
+			opt.binary = GREP_BINARY_TEXT;
+			continue;
+		}
+		if (!strcmp("-i", arg) ||
+		    !strcmp("--ignore-case", arg)) {
+			opt.regflags |= REG_ICASE;
+			continue;
+		}
+		if (!strcmp("-I", arg)) {
+			opt.binary = GREP_BINARY_NOMATCH;
+			continue;
+		}
+		if (!strcmp("-v", arg) ||
+		    !strcmp("--invert-match", arg)) {
+			opt.invert = 1;
+			continue;
+		}
+		if (!strcmp("-E", arg) ||
+		    !strcmp("--extended-regexp", arg)) {
+			opt.regflags |= REG_EXTENDED;
+			continue;
+		}
+		if (!strcmp("-G", arg) ||
+		    !strcmp("--basic-regexp", arg)) {
+			opt.regflags &= ~REG_EXTENDED;
+			continue;
+		}
+		if (!strcmp("-n", arg)) {
+			opt.linenum = 1;
+			continue;
+		}
+		if (!strcmp("-H", arg)) {
+			/* We always show the pathname, so this
+			 * is a noop.
+			 */
+			continue;
+		}
+		if (!strcmp("-l", arg) ||
+		    !strcmp("--files-with-matches", arg)) {
+			opt.name_only = 1;
+			continue;
+		}
+		if (!strcmp("-L", arg) ||
+		    !strcmp("--files-without-match", arg)) {
+			opt.unmatch_name_only = 1;
+			continue;
+		}
+		if (!strcmp("-c", arg) ||
+		    !strcmp("--count", arg)) {
+			opt.count = 1;
+			continue;
+		}
+		if (!strcmp("-w", arg) ||
+		    !strcmp("--word-regexp", arg)) {
+			opt.word_regexp = 1;
+			continue;
+		}
+		if (!strncmp("-A", arg, 2) ||
+		    !strncmp("-B", arg, 2) ||
+		    !strncmp("-C", arg, 2) ||
+		    (arg[0] == '-' && '1' <= arg[1] && arg[1] <= '9')) {
+			unsigned num;
+			const char *scan;
+			switch (arg[1]) {
+			case 'A': case 'B': case 'C':
+				if (!arg[2]) {
+					if (argc <= 1)
+						usage(builtin_grep_usage);
+					scan = *++argv;
+					argc--;
+				}
+				else
+					scan = arg + 2;
+				break;
+			default:
+				scan = arg + 1;
+				break;
+			}
+			if (sscanf(scan, "%u", &num) != 1)
+				usage(builtin_grep_usage);
+			switch (arg[1]) {
+			case 'A':
+				opt.post_context = num;
+				break;
+			default:
+			case 'C':
+				opt.post_context = num;
+			case 'B':
+				opt.pre_context = num;
+				break;
+			}
+			continue;
+		}
+		if (!strcmp("-f", arg)) {
+			FILE *patterns;
+			int lno = 0;
+			char buf[1024];
+			if (argc <= 1)
+				usage(builtin_grep_usage);
+			patterns = fopen(argv[1], "r");
+			if (!patterns)
+				die("'%s': %s", strerror(errno));
+			while (fgets(buf, sizeof(buf), patterns)) {
+				int len = strlen(buf);
+				if (buf[len-1] == '\n')
+					buf[len-1] = 0;
+				/* ignore empty line like grep does */
+				if (!buf[0])
+					continue;
+				add_pattern(&opt, strdup(buf), argv[1], ++lno);
+			}
+			fclose(patterns);
+			argv++;
+			argc--;
+			continue;
+		}
+		if (!strcmp("-e", arg)) {
+			if (1 < argc) {
+				add_pattern(&opt, argv[1], "-e option", 0);
+				argv++;
+				argc--;
+				continue;
+			}
+			usage(builtin_grep_usage);
+		}
+		if (!strcmp("--", arg)) {
+			no_more_flags = 1;
+			continue;
+		}
+		/* Either unrecognized option or a single pattern */
+		if (!no_more_flags && *arg == '-')
+			usage(builtin_grep_usage);
+		if (!opt.pattern_list) {
+			add_pattern(&opt, arg, "command line", 0);
+			break;
+		}
+		else {
+			/* We are looking at the first path or rev;
+			 * it is found at argv[0] after leaving the
+			 * loop.
+			 */
+			argc++; argv--;
+			break;
+		}
+	}
+	if (!opt.pattern_list)
+		die("no pattern given.");
+	compile_patterns(&opt);
+	tail = &object_list;
+	while (1 < argc) {
+		struct object *object;
+		struct object_list *elem;
+		const char *arg = argv[1];
+		unsigned char sha1[20];
+		if (get_sha1(arg, sha1) < 0)
+			break;
+		object = parse_object(sha1);
+		if (!object)
+			die("bad object %s", arg);
+		elem = object_list_insert(object, tail);
+		elem->name = arg;
+		tail = &elem->next;
+		argc--; argv++;
+	}
+	if (1 < argc)
+		paths = get_pathspec(prefix, argv + 1);
+	else if (prefix) {
+		paths = xcalloc(2, sizeof(const char *));
+		paths[0] = prefix;
+		paths[1] = NULL;
+	}
+
+	if (!object_list)
+		return !grep_cache(&opt, paths, cached);
+
+	if (cached)
+		die("both --cached and trees are given.");
+
+	for (list = object_list; list; list = list->next) {
+		struct object *real_obj;
+		real_obj = deref_tag(list->item, NULL, 0);
+		if (grep_object(&opt, paths, real_obj, list->name))
+			hit = 1;
+	}
+	return !hit;
+}
diff --git a/builtin-log.c b/builtin-log.c
index 69f2911cb4..d5bbc1cc06 100644
--- a/builtin-log.c
+++ b/builtin-log.c
@@ -9,6 +9,10 @@
 #include "diff.h"
 #include "revision.h"
 #include "log-tree.h"
+#include "builtin.h"
+
+/* this is in builtin-diff.c */
+void add_head(struct rev_info *revs);
 
 static int cmd_log_wc(int argc, const char **argv, char **envp,
 		      struct rev_info *rev)
@@ -67,3 +71,160 @@ int cmd_log(int argc, const char **argv, char **envp)
 	rev.diffopt.recursive = 1;
 	return cmd_log_wc(argc, argv, envp, &rev);
 }
+
+static int istitlechar(char c)
+{
+	return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') ||
+		(c >= '0' && c <= '9') || c == '.' || c == '_';
+}
+
+static FILE *realstdout = NULL;
+static char *output_directory = NULL;
+
+static void reopen_stdout(struct commit *commit, int nr, int keep_subject)
+{
+	char filename[1024];
+	char *sol;
+	int len = 0;
+
+	if (output_directory) {
+		strncpy(filename, output_directory, 1010);
+		len = strlen(filename);
+		if (filename[len - 1] != '/')
+			filename[len++] = '/';
+	}
+
+	sprintf(filename + len, "%04d", nr);
+	len = strlen(filename);
+
+	sol = strstr(commit->buffer, "\n\n");
+	if (sol) {
+		int j, space = 1;
+
+		sol += 2;
+		/* strip [PATCH] or [PATCH blabla] */
+		if (!keep_subject && !strncmp(sol, "[PATCH", 6)) {
+			char *eos = strchr(sol + 6, ']');
+			if (eos) {
+				while (isspace(*eos))
+					eos++;
+				sol = eos;
+			}
+		}
+
+		for (j = 0; len < 1024 - 6 && sol[j] && sol[j] != '\n'; j++) {
+			if (istitlechar(sol[j])) {
+				if (space) {
+					filename[len++] = '-';
+					space = 0;
+				}
+				filename[len++] = sol[j];
+				if (sol[j] == '.')
+					while (sol[j + 1] == '.')
+						j++;
+			} else
+				space = 1;
+		}
+		while (filename[len - 1] == '.' || filename[len - 1] == '-')
+			len--;
+	}
+	strcpy(filename + len, ".txt");
+	fprintf(realstdout, "%s\n", filename);
+	freopen(filename, "w", stdout);
+}
+
+int cmd_format_patch(int argc, const char **argv, char **envp)
+{
+	struct commit *commit;
+	struct commit **list = NULL;
+	struct rev_info rev;
+	int nr = 0, total, i, j;
+	int use_stdout = 0;
+	int numbered = 0;
+	int keep_subject = 0;
+
+	init_revisions(&rev);
+	rev.commit_format = CMIT_FMT_EMAIL;
+	rev.verbose_header = 1;
+	rev.diff = 1;
+	rev.diffopt.with_raw = 0;
+	rev.diffopt.with_stat = 1;
+	rev.combine_merges = 0;
+	rev.ignore_merges = 1;
+	rev.diffopt.output_format = DIFF_FORMAT_PATCH;
+
+	/*
+	 * Parse the arguments before setup_revisions(), or something
+	 * like "git fmt-patch -o a123 HEAD^.." may fail; a123 is
+	 * possibly a valid SHA1.
+	 */
+	for (i = 1, j = 1; i < argc; i++) {
+		if (!strcmp(argv[i], "--stdout"))
+			use_stdout = 1;
+		else if (!strcmp(argv[i], "-n") ||
+				!strcmp(argv[i], "--numbered"))
+			numbered = 1;
+		else if (!strcmp(argv[i], "-k") ||
+				!strcmp(argv[i], "--keep-subject")) {
+			keep_subject = 1;
+			rev.total = -1;
+		} else if (!strcmp(argv[i], "-o")) {
+			if (argc < 3)
+				die ("Which directory?");
+			if (mkdir(argv[i + 1], 0777) < 0 && errno != EEXIST)
+				die("Could not create directory %s",
+						argv[i + 1]);
+			output_directory = strdup(argv[i + 1]);
+			i++;
+		} else
+			argv[j++] = argv[i];
+	}
+	argc = j;
+
+	if (numbered && keep_subject < 0)
+		die ("-n and -k are mutually exclusive.");
+
+	argc = setup_revisions(argc, argv, &rev, "HEAD");
+	if (argc > 1)
+		die ("unrecognized argument: %s", argv[1]);
+
+	if (rev.pending_objects && rev.pending_objects->next == NULL) {
+		rev.pending_objects->item->flags |= UNINTERESTING;
+		add_head(&rev);
+	}
+
+	if (!use_stdout)
+		realstdout = fdopen(dup(1), "w");
+
+	prepare_revision_walk(&rev);
+	while ((commit = get_revision(&rev)) != NULL) {
+		/* ignore merges */
+		if (commit->parents && commit->parents->next)
+			continue;
+		nr++;
+		list = realloc(list, nr * sizeof(list[0]));
+		list[nr - 1] = commit;
+	}
+	total = nr;
+	if (numbered)
+		rev.total = total;
+	while (0 <= --nr) {
+		int shown;
+		commit = list[nr];
+		rev.nr = total - nr;
+		if (!use_stdout)
+			reopen_stdout(commit, rev.nr, keep_subject);
+		shown = log_tree_commit(&rev, commit);
+		free(commit->buffer);
+		commit->buffer = NULL;
+		if (shown)
+			printf("-- \n%s\n\n", git_version_string);
+		if (!use_stdout)
+			fclose(stdout);
+	}
+	if (output_directory)
+		free(output_directory);
+	free(list);
+	return 0;
+}
+
diff --git a/builtin.h b/builtin.h
index bb63f0729a..97e2464468 100644
--- a/builtin.h
+++ b/builtin.h
@@ -20,8 +20,10 @@ extern int cmd_whatchanged(int argc, const char **argv, char **envp);
 extern int cmd_show(int argc, const char **argv, char **envp);
 extern int cmd_log(int argc, const char **argv, char **envp);
 extern int cmd_diff(int argc, const char **argv, char **envp);
+extern int cmd_format_patch(int argc, const char **argv, char **envp);
 extern int cmd_count_objects(int argc, const char **argv, char **envp);
 
 extern int cmd_push(int argc, const char **argv, char **envp);
+extern int cmd_grep(int argc, const char **argv, char **envp);
 
 #endif
diff --git a/cache-tree.c b/cache-tree.c
new file mode 100644
index 0000000000..d9f7e1e3dd
--- /dev/null
+++ b/cache-tree.c
@@ -0,0 +1,557 @@
+#include "cache.h"
+#include "tree.h"
+#include "cache-tree.h"
+
+#define DEBUG 0
+
+struct cache_tree *cache_tree(void)
+{
+	struct cache_tree *it = xcalloc(1, sizeof(struct cache_tree));
+	it->entry_count = -1;
+	return it;
+}
+
+void cache_tree_free(struct cache_tree **it_p)
+{
+	int i;
+	struct cache_tree *it = *it_p;
+
+	if (!it)
+		return;
+	for (i = 0; i < it->subtree_nr; i++)
+		if (it->down[i])
+			cache_tree_free(&it->down[i]->cache_tree);
+	free(it->down);
+	free(it);
+	*it_p = NULL;
+}
+
+static int subtree_name_cmp(const char *one, int onelen,
+			    const char *two, int twolen)
+{
+	if (onelen < twolen)
+		return -1;
+	if (twolen < onelen)
+		return 1;
+	return memcmp(one, two, onelen);
+}
+
+static int subtree_pos(struct cache_tree *it, const char *path, int pathlen)
+{
+	struct cache_tree_sub **down = it->down;
+	int lo, hi;
+	lo = 0;
+	hi = it->subtree_nr;
+	while (lo < hi) {
+		int mi = (lo + hi) / 2;
+		struct cache_tree_sub *mdl = down[mi];
+		int cmp = subtree_name_cmp(path, pathlen,
+					   mdl->name, mdl->namelen);
+		if (!cmp)
+			return mi;
+		if (cmp < 0)
+			hi = mi;
+		else
+			lo = mi + 1;
+	}
+	return -lo-1;
+}
+
+static struct cache_tree_sub *find_subtree(struct cache_tree *it,
+					   const char *path,
+					   int pathlen,
+					   int create)
+{
+	struct cache_tree_sub *down;
+	int pos = subtree_pos(it, path, pathlen);
+	if (0 <= pos)
+		return it->down[pos];
+	if (!create)
+		return NULL;
+
+	pos = -pos-1;
+	if (it->subtree_alloc <= it->subtree_nr) {
+		it->subtree_alloc = alloc_nr(it->subtree_alloc);
+		it->down = xrealloc(it->down, it->subtree_alloc *
+				    sizeof(*it->down));
+	}
+	it->subtree_nr++;
+
+	down = xmalloc(sizeof(*down) + pathlen + 1);
+	down->cache_tree = NULL;
+	down->namelen = pathlen;
+	memcpy(down->name, path, pathlen);
+	down->name[pathlen] = 0;
+
+	if (pos < it->subtree_nr)
+		memmove(it->down + pos + 1,
+			it->down + pos,
+			sizeof(down) * (it->subtree_nr - pos - 1));
+	it->down[pos] = down;
+	return down;
+}
+
+struct cache_tree_sub *cache_tree_sub(struct cache_tree *it, const char *path)
+{
+	int pathlen = strlen(path);
+	return find_subtree(it, path, pathlen, 1);
+}
+
+void cache_tree_invalidate_path(struct cache_tree *it, const char *path)
+{
+	/* a/b/c
+	 * ==> invalidate self
+	 * ==> find "a", have it invalidate "b/c"
+	 * a
+	 * ==> invalidate self
+	 * ==> if "a" exists as a subtree, remove it.
+	 */
+	const char *slash;
+	int namelen;
+	struct cache_tree_sub *down;
+
+#if DEBUG
+	fprintf(stderr, "cache-tree invalidate <%s>\n", path);
+#endif
+
+	if (!it)
+		return;
+	slash = strchr(path, '/');
+	it->entry_count = -1;
+	if (!slash) {
+		int pos;
+		namelen = strlen(path);
+		pos = subtree_pos(it, path, namelen);
+		if (0 <= pos) {
+			cache_tree_free(&it->down[pos]->cache_tree);
+			free(it->down[pos]);
+			/* 0 1 2 3 4 5
+			 *       ^     ^subtree_nr = 6
+			 *       pos
+			 * move 4 and 5 up one place (2 entries)
+			 * 2 = 6 - 3 - 1 = subtree_nr - pos - 1
+			 */
+			memmove(it->down+pos, it->down+pos+1,
+				sizeof(struct cache_tree_sub *) *
+				(it->subtree_nr - pos - 1));
+			it->subtree_nr--;
+		}
+		return;
+	}
+	namelen = slash - path;
+	down = find_subtree(it, path, namelen, 0);
+	if (down)
+		cache_tree_invalidate_path(down->cache_tree, slash + 1);
+}
+
+static int verify_cache(struct cache_entry **cache,
+			int entries)
+{
+	int i, funny;
+
+	/* Verify that the tree is merged */
+	funny = 0;
+	for (i = 0; i < entries; i++) {
+		struct cache_entry *ce = cache[i];
+		if (ce_stage(ce)) {
+			if (10 < ++funny) {
+				fprintf(stderr, "...\n");
+				break;
+			}
+			fprintf(stderr, "%s: unmerged (%s)\n",
+				ce->name, sha1_to_hex(ce->sha1));
+		}
+	}
+	if (funny)
+		return -1;
+
+	/* Also verify that the cache does not have path and path/file
+	 * at the same time.  At this point we know the cache has only
+	 * stage 0 entries.
+	 */
+	funny = 0;
+	for (i = 0; i < entries - 1; i++) {
+		/* path/file always comes after path because of the way
+		 * the cache is sorted.  Also path can appear only once,
+		 * which means conflicting one would immediately follow.
+		 */
+		const char *this_name = cache[i]->name;
+		const char *next_name = cache[i+1]->name;
+		int this_len = strlen(this_name);
+		if (this_len < strlen(next_name) &&
+		    strncmp(this_name, next_name, this_len) == 0 &&
+		    next_name[this_len] == '/') {
+			if (10 < ++funny) {
+				fprintf(stderr, "...\n");
+				break;
+			}
+			fprintf(stderr, "You have both %s and %s\n",
+				this_name, next_name);
+		}
+	}
+	if (funny)
+		return -1;
+	return 0;
+}
+
+static void discard_unused_subtrees(struct cache_tree *it)
+{
+	struct cache_tree_sub **down = it->down;
+	int nr = it->subtree_nr;
+	int dst, src;
+	for (dst = src = 0; src < nr; src++) {
+		struct cache_tree_sub *s = down[src];
+		if (s->used)
+			down[dst++] = s;
+		else {
+			cache_tree_free(&s->cache_tree);
+			free(s);
+			it->subtree_nr--;
+		}
+	}
+}
+
+int cache_tree_fully_valid(struct cache_tree *it)
+{
+	int i;
+	if (!it)
+		return 0;
+	if (it->entry_count < 0 || !has_sha1_file(it->sha1))
+		return 0;
+	for (i = 0; i < it->subtree_nr; i++) {
+		if (!cache_tree_fully_valid(it->down[i]->cache_tree))
+			return 0;
+	}
+	return 1;
+}
+
+static int update_one(struct cache_tree *it,
+		      struct cache_entry **cache,
+		      int entries,
+		      const char *base,
+		      int baselen,
+		      int missing_ok,
+		      int dryrun)
+{
+	unsigned long size, offset;
+	char *buffer;
+	int i;
+
+	if (0 <= it->entry_count && has_sha1_file(it->sha1))
+		return it->entry_count;
+
+	/*
+	 * We first scan for subtrees and update them; we start by
+	 * marking existing subtrees -- the ones that are unmarked
+	 * should not be in the result.
+	 */
+	for (i = 0; i < it->subtree_nr; i++)
+		it->down[i]->used = 0;
+
+	/*
+	 * Find the subtrees and update them.
+	 */
+	for (i = 0; i < entries; i++) {
+		struct cache_entry *ce = cache[i];
+		struct cache_tree_sub *sub;
+		const char *path, *slash;
+		int pathlen, sublen, subcnt;
+
+		path = ce->name;
+		pathlen = ce_namelen(ce);
+		if (pathlen <= baselen || memcmp(base, path, baselen))
+			break; /* at the end of this level */
+
+		slash = strchr(path + baselen, '/');
+		if (!slash)
+			continue;
+		/*
+		 * a/bbb/c (base = a/, slash = /c)
+		 * ==>
+		 * path+baselen = bbb/c, sublen = 3
+		 */
+		sublen = slash - (path + baselen);
+		sub = find_subtree(it, path + baselen, sublen, 1);
+		if (!sub->cache_tree)
+			sub->cache_tree = cache_tree();
+		subcnt = update_one(sub->cache_tree,
+				    cache + i, entries - i,
+				    path,
+				    baselen + sublen + 1,
+				    missing_ok,
+				    dryrun);
+		i += subcnt - 1;
+		sub->used = 1;
+	}
+
+	discard_unused_subtrees(it);
+
+	/*
+	 * Then write out the tree object for this level.
+	 */
+	size = 8192;
+	buffer = xmalloc(size);
+	offset = 0;
+
+	for (i = 0; i < entries; i++) {
+		struct cache_entry *ce = cache[i];
+		struct cache_tree_sub *sub;
+		const char *path, *slash;
+		int pathlen, entlen;
+		const unsigned char *sha1;
+		unsigned mode;
+
+		path = ce->name;
+		pathlen = ce_namelen(ce);
+		if (pathlen <= baselen || memcmp(base, path, baselen))
+			break; /* at the end of this level */
+
+		slash = strchr(path + baselen, '/');
+		if (slash) {
+			entlen = slash - (path + baselen);
+			sub = find_subtree(it, path + baselen, entlen, 0);
+			if (!sub)
+				die("cache-tree.c: '%.*s' in '%s' not found",
+				    entlen, path + baselen, path);
+			i += sub->cache_tree->entry_count - 1;
+			sha1 = sub->cache_tree->sha1;
+			mode = S_IFDIR;
+		}
+		else {
+			sha1 = ce->sha1;
+			mode = ntohl(ce->ce_mode);
+			entlen = pathlen - baselen;
+		}
+		if (!missing_ok && !has_sha1_file(sha1))
+			return error("invalid object %s", sha1_to_hex(sha1));
+
+		if (!ce->ce_mode)
+			continue; /* entry being removed */
+
+		if (size < offset + entlen + 100) {
+			size = alloc_nr(offset + entlen + 100);
+			buffer = xrealloc(buffer, size);
+		}
+		offset += sprintf(buffer + offset,
+				  "%o %.*s", mode, entlen, path + baselen);
+		buffer[offset++] = 0;
+		memcpy(buffer + offset, sha1, 20);
+		offset += 20;
+
+#if DEBUG
+		fprintf(stderr, "cache-tree update-one %o %.*s\n",
+			mode, entlen, path + baselen);
+#endif
+	}
+
+	if (dryrun) {
+		unsigned char hdr[200];
+		int hdrlen;
+		write_sha1_file_prepare(buffer, offset, tree_type, it->sha1,
+					hdr, &hdrlen);
+	}
+	else
+		write_sha1_file(buffer, offset, tree_type, it->sha1);
+	free(buffer);
+	it->entry_count = i;
+#if DEBUG
+	fprintf(stderr, "cache-tree update-one (%d ent, %d subtree) %s\n",
+		it->entry_count, it->subtree_nr,
+		sha1_to_hex(it->sha1));
+#endif
+	return i;
+}
+
+int cache_tree_update(struct cache_tree *it,
+		      struct cache_entry **cache,
+		      int entries,
+		      int missing_ok,
+		      int dryrun)
+{
+	int i;
+	i = verify_cache(cache, entries);
+	if (i)
+		return i;
+	i = update_one(it, cache, entries, "", 0, missing_ok, dryrun);
+	if (i < 0)
+		return i;
+	return 0;
+}
+
+static void *write_one(struct cache_tree *it,
+		       char *path,
+		       int pathlen,
+		       char *buffer,
+		       unsigned long *size,
+		       unsigned long *offset)
+{
+	int i;
+
+	/* One "cache-tree" entry consists of the following:
+	 * path (NUL terminated)
+	 * entry_count, subtree_nr ("%d %d\n")
+	 * tree-sha1 (missing if invalid)
+	 * subtree_nr "cache-tree" entries for subtrees.
+	 */
+	if (*size < *offset + pathlen + 100) {
+		*size = alloc_nr(*offset + pathlen + 100);
+		buffer = xrealloc(buffer, *size);
+	}
+	*offset += sprintf(buffer + *offset, "%.*s%c%d %d\n",
+			   pathlen, path, 0,
+			   it->entry_count, it->subtree_nr);
+
+#if DEBUG
+	if (0 <= it->entry_count)
+		fprintf(stderr, "cache-tree <%.*s> (%d ent, %d subtree) %s\n",
+			pathlen, path, it->entry_count, it->subtree_nr,
+			sha1_to_hex(it->sha1));
+	else
+		fprintf(stderr, "cache-tree <%.*s> (%d subtree) invalid\n",
+			pathlen, path, it->subtree_nr);
+#endif
+
+	if (0 <= it->entry_count) {
+		memcpy(buffer + *offset, it->sha1, 20);
+		*offset += 20;
+	}
+	for (i = 0; i < it->subtree_nr; i++) {
+		struct cache_tree_sub *down = it->down[i];
+		if (i) {
+			struct cache_tree_sub *prev = it->down[i-1];
+			if (subtree_name_cmp(down->name, down->namelen,
+					     prev->name, prev->namelen) <= 0)
+				die("fatal - unsorted cache subtree");
+		}
+		buffer = write_one(down->cache_tree, down->name, down->namelen,
+				   buffer, size, offset);
+	}
+	return buffer;
+}
+
+void *cache_tree_write(struct cache_tree *root, unsigned long *size_p)
+{
+	char path[PATH_MAX];
+	unsigned long size = 8192;
+	char *buffer = xmalloc(size);
+
+	*size_p = 0;
+	path[0] = 0;
+	return write_one(root, path, 0, buffer, &size, size_p);
+}
+
+static struct cache_tree *read_one(const char **buffer, unsigned long *size_p)
+{
+	const char *buf = *buffer;
+	unsigned long size = *size_p;
+	const char *cp;
+	char *ep;
+	struct cache_tree *it;
+	int i, subtree_nr;
+
+	it = NULL;
+	/* skip name, but make sure name exists */
+	while (size && *buf) {
+		size--;
+		buf++;
+	}
+	if (!size)
+		goto free_return;
+	buf++; size--;
+	it = cache_tree();
+
+	cp = buf;
+	it->entry_count = strtol(cp, &ep, 10);
+	if (cp == ep)
+		goto free_return;
+	cp = ep;
+	subtree_nr = strtol(cp, &ep, 10);
+	if (cp == ep)
+		goto free_return;
+	while (size && *buf && *buf != '\n') {
+		size--;
+		buf++;
+	}
+	if (!size)
+		goto free_return;
+	buf++; size--;
+	if (0 <= it->entry_count) {
+		if (size < 20)
+			goto free_return;
+		memcpy(it->sha1, buf, 20);
+		buf += 20;
+		size -= 20;
+	}
+
+#if DEBUG
+	if (0 <= it->entry_count)
+		fprintf(stderr, "cache-tree <%s> (%d ent, %d subtree) %s\n",
+			*buffer, it->entry_count, subtree_nr,
+			sha1_to_hex(it->sha1));
+	else
+		fprintf(stderr, "cache-tree <%s> (%d subtrees) invalid\n",
+			*buffer, subtree_nr);
+#endif
+
+	/*
+	 * Just a heuristic -- we do not add directories that often but
+	 * we do not want to have to extend it immediately when we do,
+	 * hence +2.
+	 */
+	it->subtree_alloc = subtree_nr + 2;
+	it->down = xcalloc(it->subtree_alloc, sizeof(struct cache_tree_sub *));
+	for (i = 0; i < subtree_nr; i++) {
+		/* read each subtree */
+		struct cache_tree *sub;
+		struct cache_tree_sub *subtree;
+		const char *name = buf;
+
+		sub = read_one(&buf, &size);
+		if (!sub)
+			goto free_return;
+		subtree = cache_tree_sub(it, name);
+		subtree->cache_tree = sub;
+	}
+	if (subtree_nr != it->subtree_nr)
+		die("cache-tree: internal error");
+	*buffer = buf;
+	*size_p = size;
+	return it;
+
+ free_return:
+	cache_tree_free(&it);
+	return NULL;
+}
+
+struct cache_tree *cache_tree_read(const char *buffer, unsigned long size)
+{
+	if (buffer[0])
+		return NULL; /* not the whole tree */
+	return read_one(&buffer, &size);
+}
+
+struct cache_tree *cache_tree_find(struct cache_tree *it, const char *path)
+{
+	while (*path) {
+		const char *slash;
+		struct cache_tree_sub *sub;
+
+		slash = strchr(path, '/');
+		if (!slash)
+			slash = path + strlen(path);
+		/* between path and slash is the name of the
+		 * subtree to look for.
+		 */
+		sub = find_subtree(it, path, slash - path, 0);
+		if (!sub)
+			return NULL;
+		it = sub->cache_tree;
+		if (slash)
+			while (*slash && *slash == '/')
+				slash++;
+		if (!slash || !*slash)
+			return it; /* prefix ended with slashes */
+		path = slash;
+	}
+	return it;
+}
diff --git a/cache-tree.h b/cache-tree.h
new file mode 100644
index 0000000000..119407e3a1
--- /dev/null
+++ b/cache-tree.h
@@ -0,0 +1,33 @@
+#ifndef CACHE_TREE_H
+#define CACHE_TREE_H
+
+struct cache_tree;
+struct cache_tree_sub {
+	struct cache_tree *cache_tree;
+	int namelen;
+	int used;
+	char name[FLEX_ARRAY];
+};
+
+struct cache_tree {
+	int entry_count; /* negative means "invalid" */
+	unsigned char sha1[20];
+	int subtree_nr;
+	int subtree_alloc;
+	struct cache_tree_sub **down;
+};
+
+struct cache_tree *cache_tree(void);
+void cache_tree_free(struct cache_tree **);
+void cache_tree_invalidate_path(struct cache_tree *, const char *);
+struct cache_tree_sub *cache_tree_sub(struct cache_tree *, const char *);
+
+void *cache_tree_write(struct cache_tree *root, unsigned long *size_p);
+struct cache_tree *cache_tree_read(const char *buffer, unsigned long size);
+
+int cache_tree_fully_valid(struct cache_tree *);
+int cache_tree_update(struct cache_tree *, struct cache_entry **, int, int, int);
+
+struct cache_tree *cache_tree_find(struct cache_tree *, const char *);
+
+#endif
diff --git a/cache.h b/cache.h
index 4b7a439253..b1300cd989 100644
--- a/cache.h
+++ b/cache.h
@@ -114,6 +114,7 @@ static inline unsigned int create_ce_mode(unsigned int mode)
 
 extern struct cache_entry **active_cache;
 extern unsigned int active_nr, active_alloc, active_cache_changed;
+extern struct cache_tree *active_cache_tree;
 
 #define GIT_DIR_ENVIRONMENT "GIT_DIR"
 #define DEFAULT_GIT_DIR_ENVIRONMENT ".git"
@@ -251,6 +252,7 @@ extern void *read_object_with_reference(const unsigned char *sha1,
 					unsigned char *sha1_ret);
 
 const char *show_date(unsigned long time, int timezone);
+const char *show_rfc2822_date(unsigned long time, int timezone);
 int parse_date(const char *date, char *buf, int bufsize);
 void datestamp(char *buf, int bufsize);
 unsigned long approxidate(const char *);
diff --git a/checkout-index.c b/checkout-index.c
index dd6a2d86fe..9876af6fd6 100644
--- a/checkout-index.c
+++ b/checkout-index.c
@@ -39,6 +39,7 @@
 #include "cache.h"
 #include "strbuf.h"
 #include "quote.h"
+#include "cache-tree.h"
 
 #define CHECKOUT_ALL 4
 static const char *prefix;
@@ -269,12 +270,16 @@ int main(int argc, char **argv)
 	/* Check out named files first */
 	for ( ; i < argc; i++) {
 		const char *arg = argv[i];
+		const char *p;
 
 		if (all)
 			die("git-checkout-index: don't mix '--all' and explicit filenames");
 		if (read_from_stdin)
 			die("git-checkout-index: don't mix '--stdin' and explicit filenames");
-		checkout_file(prefix_path(prefix, prefix_length, arg));
+		p = prefix_path(prefix, prefix_length, arg);
+		checkout_file(p);
+		if (p < arg || p > arg + strlen(arg))
+			free((char*)p);
 	}
 
 	if (read_from_stdin) {
@@ -284,6 +289,8 @@ int main(int argc, char **argv)
 		strbuf_init(&buf);
 		while (1) {
 			char *path_name;
+			const char *p;
+
 			read_line(&buf, stdin, line_termination);
 			if (buf.eof)
 				break;
@@ -291,7 +298,10 @@ int main(int argc, char **argv)
 				path_name = unquote_c_style(buf.buf, NULL);
 			else
 				path_name = buf.buf;
-			checkout_file(prefix_path(prefix, prefix_length, path_name));
+			p = prefix_path(prefix, prefix_length, path_name);
+			checkout_file(p);
+			if (p < path_name || p > path_name + strlen(path_name))
+				free((char *)p);
 			if (path_name != buf.buf)
 				free(path_name);
 		}
diff --git a/commit.c b/commit.c
index 2717dd81c3..93b3903ea7 100644
--- a/commit.c
+++ b/commit.c
@@ -36,6 +36,8 @@ enum cmit_fmt get_commit_format(const char *arg)
 		return CMIT_FMT_FULL;
 	if (!strcmp(arg, "=fuller"))
 		return CMIT_FMT_FULLER;
+	if (!strcmp(arg, "=email"))
+		return CMIT_FMT_EMAIL;
 	if (!strcmp(arg, "=oneline"))
 		return CMIT_FMT_ONELINE;
 	die("invalid --pretty format");
@@ -428,6 +430,10 @@ static int add_user_info(const char *what, enum cmit_fmt fmt, char *buf, const c
 	time = strtoul(date, &date, 10);
 	tz = strtol(date, NULL, 10);
 
+	if (fmt == CMIT_FMT_EMAIL) {
+		what = "From";
+		filler = "";
+	}
 	ret = sprintf(buf, "%s: %.*s%.*s\n", what,
 		      (fmt == CMIT_FMT_FULLER) ? 4 : 0,
 		      filler, namelen, line);
@@ -435,6 +441,10 @@ static int add_user_info(const char *what, enum cmit_fmt fmt, char *buf, const c
 	case CMIT_FMT_MEDIUM:
 		ret += sprintf(buf + ret, "Date:   %s\n", show_date(time, tz));
 		break;
+	case CMIT_FMT_EMAIL:
+		ret += sprintf(buf + ret, "Date: %s\n",
+			       show_rfc2822_date(time, tz));
+		break;
 	case CMIT_FMT_FULLER:
 		ret += sprintf(buf + ret, "%sDate: %s\n", what, show_date(time, tz));
 		break;
@@ -445,10 +455,12 @@ static int add_user_info(const char *what, enum cmit_fmt fmt, char *buf, const c
 	return ret;
 }
 
-static int is_empty_line(const char *line, int len)
+static int is_empty_line(const char *line, int *len_p)
 {
+	int len = *len_p;
 	while (len && isspace(line[len-1]))
 		len--;
+	*len_p = len;
 	return !len;
 }
 
@@ -457,7 +469,8 @@ static int add_merge_info(enum cmit_fmt fmt, char *buf, const struct commit *com
 	struct commit_list *parent = commit->parents;
 	int offset;
 
-	if ((fmt == CMIT_FMT_ONELINE) || !parent || !parent->next)
+	if ((fmt == CMIT_FMT_ONELINE) || (fmt == CMIT_FMT_EMAIL) ||
+	    !parent || !parent->next)
 		return 0;
 
 	offset = sprintf(buf, "Merge:");
@@ -476,14 +489,17 @@ static int add_merge_info(enum cmit_fmt fmt, char *buf, const struct commit *com
 	return offset;
 }
 
-unsigned long pretty_print_commit(enum cmit_fmt fmt, const struct commit *commit, unsigned long len, char *buf, unsigned long space, int abbrev)
+unsigned long pretty_print_commit(enum cmit_fmt fmt, const struct commit *commit, unsigned long len, char *buf, unsigned long space, int abbrev, const char *subject)
 {
 	int hdr = 1, body = 0;
 	unsigned long offset = 0;
-	int indent = (fmt == CMIT_FMT_ONELINE) ? 0 : 4;
+	int indent = 4;
 	int parents_shown = 0;
 	const char *msg = commit->buffer;
 
+	if (fmt == CMIT_FMT_ONELINE || fmt == CMIT_FMT_EMAIL)
+		indent = 0;
+
 	for (;;) {
 		const char *line = msg;
 		int linelen = get_one_line(msg, len);
@@ -506,7 +522,7 @@ unsigned long pretty_print_commit(enum cmit_fmt fmt, const struct commit *commit
 		if (hdr) {
 			if (linelen == 1) {
 				hdr = 0;
-				if (fmt != CMIT_FMT_ONELINE)
+				if ((fmt != CMIT_FMT_ONELINE) && !subject)
 					buf[offset++] = '\n';
 				continue;
 			}
@@ -544,20 +560,29 @@ unsigned long pretty_print_commit(enum cmit_fmt fmt, const struct commit *commit
 			continue;
 		}
 
-		if (is_empty_line(line, linelen)) {
+		if (is_empty_line(line, &linelen)) {
 			if (!body)
 				continue;
+			if (subject)
+				continue;
 			if (fmt == CMIT_FMT_SHORT)
 				break;
 		} else {
 			body = 1;
 		}
 
+		if (subject) {
+			int slen = strlen(subject);
+			memcpy(buf + offset, subject, slen);
+			offset += slen;
+		}
 		memset(buf + offset, ' ', indent);
 		memcpy(buf + offset + indent, line, linelen);
 		offset += linelen + indent;
+		buf[offset++] = '\n';
 		if (fmt == CMIT_FMT_ONELINE)
 			break;
+		subject = NULL;
 	}
 	while (offset && isspace(buf[offset-1]))
 		offset--;
diff --git a/commit.h b/commit.h
index de142afe73..8d7514cd00 100644
--- a/commit.h
+++ b/commit.h
@@ -45,12 +45,13 @@ enum cmit_fmt {
 	CMIT_FMT_FULL,
 	CMIT_FMT_FULLER,
 	CMIT_FMT_ONELINE,
+	CMIT_FMT_EMAIL,
 
 	CMIT_FMT_UNSPECIFIED,
 };
 
 extern enum cmit_fmt get_commit_format(const char *arg);
-extern unsigned long pretty_print_commit(enum cmit_fmt fmt, const struct commit *, unsigned long len, char *buf, unsigned long space, int abbrev);
+extern unsigned long pretty_print_commit(enum cmit_fmt fmt, const struct commit *, unsigned long len, char *buf, unsigned long space, int abbrev, const char *subject);
 
 /** Removes the first commit from a list sorted by date, and adds all
  * of its parents.
diff --git a/config.c b/config.c
index 87fb22041e..61eacefbee 100644
--- a/config.c
+++ b/config.c
@@ -336,8 +336,10 @@ static int store_aux(const char* key, const char* value)
 			store.state = KEY_SEEN;
 			store.seen++;
 		} else if (strrchr(key, '.') - key == store.baselen &&
-			      !strncmp(key, store.key, store.baselen))
+			      !strncmp(key, store.key, store.baselen)) {
 					store.state = SECTION_SEEN;
+					store.offset[store.seen] = ftell(config_file);
+		}
 	}
 	return 0;
 }
@@ -514,6 +516,8 @@ int git_config_set_multivar(const char* key, const char* value,
 				fprintf(stderr, "Invalid pattern: %s\n",
 					value_regex);
 				free(store.value_regex);
+				close(fd);
+				unlink(lock_file);
 				ret = 6;
 				goto out_free;
 			}
diff --git a/contrib/remotes2config.sh b/contrib/remotes2config.sh
new file mode 100644
index 0000000000..25901e2b3b
--- /dev/null
+++ b/contrib/remotes2config.sh
@@ -0,0 +1,35 @@
+#!/bin/sh
+
+# Use this tool to rewrite your .git/remotes/ files into the config.
+
+. git-sh-setup
+
+if [ -d "$GIT_DIR"/remotes ]; then
+	echo "Rewriting $GIT_DIR/remotes" >&2
+	error=0
+	# rewrite into config
+	{
+		cd "$GIT_DIR"/remotes
+		ls | while read f; do
+			name=$(echo -n "$f" | tr -c "A-Za-z0-9" ".")
+			sed -n \
+			-e "s/^URL: \(.*\)$/remote.$name.url \1 ./p" \
+			-e "s/^Pull: \(.*\)$/remote.$name.fetch \1 ^$ /p" \
+			-e "s/^Push: \(.*\)$/remote.$name.push \1 ^$ /p" \
+			< "$f"
+		done
+		echo done
+	} | while read key value regex; do
+		case $key in
+		done)
+			if [ $error = 0 ]; then
+				mv "$GIT_DIR"/remotes "$GIT_DIR"/remotes.old
+			fi ;;
+		*)
+			echo "git-repo-config $key "$value" $regex"
+			git-repo-config $key "$value" $regex || error=1 ;;
+		esac
+	done
+fi
+
+
diff --git a/date.c b/date.c
index 034d7228bf..365dc3b14e 100644
--- a/date.c
+++ b/date.c
@@ -42,18 +42,24 @@ static const char *weekday_names[] = {
  * thing, which means that tz -0100 is passed in as the integer -100,
  * even though it means "sixty minutes off"
  */
-const char *show_date(unsigned long time, int tz)
+static struct tm *time_to_tm(unsigned long time, int tz)
 {
-	struct tm *tm;
 	time_t t;
-	static char timebuf[200];
 	int minutes;
 
 	minutes = tz < 0 ? -tz : tz;
 	minutes = (minutes / 100)*60 + (minutes % 100);
 	minutes = tz < 0 ? -minutes : minutes;
 	t = time + minutes * 60;
-	tm = gmtime(&t);
+	return gmtime(&t);
+}
+
+const char *show_date(unsigned long time, int tz)
+{
+	struct tm *tm;
+	static char timebuf[200];
+
+	tm = time_to_tm(time, tz);
 	if (!tm)
 		return NULL;
 	sprintf(timebuf, "%.3s %.3s %d %02d:%02d:%02d %d %+05d",
@@ -65,6 +71,21 @@ const char *show_date(unsigned long time, int tz)
 	return timebuf;
 }
 
+const char *show_rfc2822_date(unsigned long time, int tz)
+{
+	struct tm *tm;
+	static char timebuf[200];
+
+	tm = time_to_tm(time, tz);
+	if (!tm)
+		return NULL;
+	sprintf(timebuf, "%.3s, %d %.3s %d %02d:%02d:%02d %+05d",
+		weekday_names[tm->tm_wday], tm->tm_mday,
+		month_names[tm->tm_mon], tm->tm_year + 1900,
+		tm->tm_hour, tm->tm_min, tm->tm_sec, tz);
+	return timebuf;
+}
+
 /*
  * Check these. And note how it doesn't do the summer-time conversion.
  *
diff --git a/delta.h b/delta.h
index 09b2e1eed5..727ae30e9e 100644
--- a/delta.h
+++ b/delta.h
@@ -1,11 +1,72 @@
 #ifndef DELTA_H
 #define DELTA_H
 
-/* handling of delta buffers */
-extern void *diff_delta(const void *from_buf, unsigned long from_size,
-			const void *to_buf, unsigned long to_size,
-		        unsigned long *delta_size, unsigned long max_size);
-extern void *patch_delta(void *src_buf, unsigned long src_size,
+/* opaque object for delta index */
+struct delta_index;
+
+/*
+ * create_delta_index: compute index data from given buffer
+ *
+ * This returns a pointer to a struct delta_index that should be passed to
+ * subsequent create_delta() calls, or to free_delta_index().  A NULL pointer
+ * is returned on failure.  The given buffer must not be freed nor altered
+ * before free_delta_index() is called.  The returned pointer must be freed
+ * using free_delta_index().
+ */
+extern struct delta_index *
+create_delta_index(const void *buf, unsigned long bufsize);
+
+/*
+ * free_delta_index: free the index created by create_delta_index()
+ */
+extern void free_delta_index(struct delta_index *index);
+
+/*
+ * create_delta: create a delta from given index for the given buffer
+ *
+ * This function may be called multiple times with different buffers using
+ * the same delta_index pointer.  If max_delta_size is non-zero and the
+ * resulting delta is to be larger than max_delta_size then NULL is returned.
+ * On success, a non-NULL pointer to the buffer with the delta data is
+ * returned and *delta_size is updated with its size.  The returned buffer
+ * must be freed by the caller.
+ */
+extern void *
+create_delta(const struct delta_index *index,
+	     const void *buf, unsigned long bufsize,
+	     unsigned long *delta_size, unsigned long max_delta_size);
+
+/*
+ * diff_delta: create a delta from source buffer to target buffer
+ *
+ * If max_delta_size is non-zero and the resulting delta is to be larger
+ * than max_delta_size then NULL is returned.  On success, a non-NULL
+ * pointer to the buffer with the delta data is returned and *delta_size is
+ * updated with its size.  The returned buffer must be freed by the caller.
+ */
+static inline void *
+diff_delta(const void *src_buf, unsigned long src_bufsize,
+	   const void *trg_buf, unsigned long trg_bufsize,
+	   unsigned long *delta_size, unsigned long max_delta_size)
+{
+	struct delta_index *index = create_delta_index(src_buf, src_bufsize);
+	if (index) {
+		void *delta = create_delta(index, trg_buf, trg_bufsize,
+					   delta_size, max_delta_size);
+		free_delta_index(index);
+		return delta;
+	}
+	return NULL;
+}
+
+/*
+ * patch_delta: recreate target buffer given source buffer and delta data
+ *
+ * On success, a non-NULL pointer to the target buffer is returned and
+ * *trg_bufsize is updated with its size.  On failure a NULL pointer is
+ * returned.  The returned buffer must be freed by the caller.
+ */
+extern void *patch_delta(const void *src_buf, unsigned long src_size,
 			 const void *delta_buf, unsigned long delta_size,
 			 unsigned long *dst_size);
 
@@ -14,7 +75,7 @@ extern void *patch_delta(void *src_buf, unsigned long src_size,
 
 /*
  * This must be called twice on the delta data buffer, first to get the
- * expected reference buffer size, and again to get the result buffer size.
+ * expected source buffer size, and again to get the target buffer size.
  */
 static inline unsigned long get_delta_hdr_size(const unsigned char **datap,
 					       const unsigned char *top)
diff --git a/diff-delta.c b/diff-delta.c
index 5bc22047f8..c618875188 100644
--- a/diff-delta.c
+++ b/diff-delta.c
@@ -20,69 +20,188 @@
 
 #include <stdlib.h>
 #include <string.h>
-#include <zlib.h>
 #include "delta.h"
 
 
-/* block size: min = 16, max = 64k, power of 2 */
-#define BLK_SIZE 16
-
-#define MIN(a, b) ((a) < (b) ? (a) : (b))
+/* maximum hash entry list for the same hash bucket */
+#define HASH_LIMIT 64
+
+#define RABIN_SHIFT 23
+#define RABIN_WINDOW 16
+
+static const unsigned int T[256] = {
+	0x00000000, 0xab59b4d1, 0x56b369a2, 0xfdeadd73, 0x063f6795, 0xad66d344,
+	0x508c0e37, 0xfbd5bae6, 0x0c7ecf2a, 0xa7277bfb, 0x5acda688, 0xf1941259,
+	0x0a41a8bf, 0xa1181c6e, 0x5cf2c11d, 0xf7ab75cc, 0x18fd9e54, 0xb3a42a85,
+	0x4e4ef7f6, 0xe5174327, 0x1ec2f9c1, 0xb59b4d10, 0x48719063, 0xe32824b2,
+	0x1483517e, 0xbfdae5af, 0x423038dc, 0xe9698c0d, 0x12bc36eb, 0xb9e5823a,
+	0x440f5f49, 0xef56eb98, 0x31fb3ca8, 0x9aa28879, 0x6748550a, 0xcc11e1db,
+	0x37c45b3d, 0x9c9defec, 0x6177329f, 0xca2e864e, 0x3d85f382, 0x96dc4753,
+	0x6b369a20, 0xc06f2ef1, 0x3bba9417, 0x90e320c6, 0x6d09fdb5, 0xc6504964,
+	0x2906a2fc, 0x825f162d, 0x7fb5cb5e, 0xd4ec7f8f, 0x2f39c569, 0x846071b8,
+	0x798aaccb, 0xd2d3181a, 0x25786dd6, 0x8e21d907, 0x73cb0474, 0xd892b0a5,
+	0x23470a43, 0x881ebe92, 0x75f463e1, 0xdeadd730, 0x63f67950, 0xc8afcd81,
+	0x354510f2, 0x9e1ca423, 0x65c91ec5, 0xce90aa14, 0x337a7767, 0x9823c3b6,
+	0x6f88b67a, 0xc4d102ab, 0x393bdfd8, 0x92626b09, 0x69b7d1ef, 0xc2ee653e,
+	0x3f04b84d, 0x945d0c9c, 0x7b0be704, 0xd05253d5, 0x2db88ea6, 0x86e13a77,
+	0x7d348091, 0xd66d3440, 0x2b87e933, 0x80de5de2, 0x7775282e, 0xdc2c9cff,
+	0x21c6418c, 0x8a9ff55d, 0x714a4fbb, 0xda13fb6a, 0x27f92619, 0x8ca092c8,
+	0x520d45f8, 0xf954f129, 0x04be2c5a, 0xafe7988b, 0x5432226d, 0xff6b96bc,
+	0x02814bcf, 0xa9d8ff1e, 0x5e738ad2, 0xf52a3e03, 0x08c0e370, 0xa39957a1,
+	0x584ced47, 0xf3155996, 0x0eff84e5, 0xa5a63034, 0x4af0dbac, 0xe1a96f7d,
+	0x1c43b20e, 0xb71a06df, 0x4ccfbc39, 0xe79608e8, 0x1a7cd59b, 0xb125614a,
+	0x468e1486, 0xedd7a057, 0x103d7d24, 0xbb64c9f5, 0x40b17313, 0xebe8c7c2,
+	0x16021ab1, 0xbd5bae60, 0x6cb54671, 0xc7ecf2a0, 0x3a062fd3, 0x915f9b02,
+	0x6a8a21e4, 0xc1d39535, 0x3c394846, 0x9760fc97, 0x60cb895b, 0xcb923d8a,
+	0x3678e0f9, 0x9d215428, 0x66f4eece, 0xcdad5a1f, 0x3047876c, 0x9b1e33bd,
+	0x7448d825, 0xdf116cf4, 0x22fbb187, 0x89a20556, 0x7277bfb0, 0xd92e0b61,
+	0x24c4d612, 0x8f9d62c3, 0x7836170f, 0xd36fa3de, 0x2e857ead, 0x85dcca7c,
+	0x7e09709a, 0xd550c44b, 0x28ba1938, 0x83e3ade9, 0x5d4e7ad9, 0xf617ce08,
+	0x0bfd137b, 0xa0a4a7aa, 0x5b711d4c, 0xf028a99d, 0x0dc274ee, 0xa69bc03f,
+	0x5130b5f3, 0xfa690122, 0x0783dc51, 0xacda6880, 0x570fd266, 0xfc5666b7,
+	0x01bcbbc4, 0xaae50f15, 0x45b3e48d, 0xeeea505c, 0x13008d2f, 0xb85939fe,
+	0x438c8318, 0xe8d537c9, 0x153feaba, 0xbe665e6b, 0x49cd2ba7, 0xe2949f76,
+	0x1f7e4205, 0xb427f6d4, 0x4ff24c32, 0xe4abf8e3, 0x19412590, 0xb2189141,
+	0x0f433f21, 0xa41a8bf0, 0x59f05683, 0xf2a9e252, 0x097c58b4, 0xa225ec65,
+	0x5fcf3116, 0xf49685c7, 0x033df00b, 0xa86444da, 0x558e99a9, 0xfed72d78,
+	0x0502979e, 0xae5b234f, 0x53b1fe3c, 0xf8e84aed, 0x17bea175, 0xbce715a4,
+	0x410dc8d7, 0xea547c06, 0x1181c6e0, 0xbad87231, 0x4732af42, 0xec6b1b93,
+	0x1bc06e5f, 0xb099da8e, 0x4d7307fd, 0xe62ab32c, 0x1dff09ca, 0xb6a6bd1b,
+	0x4b4c6068, 0xe015d4b9, 0x3eb80389, 0x95e1b758, 0x680b6a2b, 0xc352defa,
+	0x3887641c, 0x93ded0cd, 0x6e340dbe, 0xc56db96f, 0x32c6cca3, 0x999f7872,
+	0x6475a501, 0xcf2c11d0, 0x34f9ab36, 0x9fa01fe7, 0x624ac294, 0xc9137645,
+	0x26459ddd, 0x8d1c290c, 0x70f6f47f, 0xdbaf40ae, 0x207afa48, 0x8b234e99,
+	0x76c993ea, 0xdd90273b, 0x2a3b52f7, 0x8162e626, 0x7c883b55, 0xd7d18f84,
+	0x2c043562, 0x875d81b3, 0x7ab75cc0, 0xd1eee811
+};
 
-#define GR_PRIME 0x9e370001
-#define HASH(v, shift) (((unsigned int)(v) * GR_PRIME) >> (shift))
+static const unsigned int U[256] = {
+	0x00000000, 0x7eb5200d, 0x5633f4cb, 0x2886d4c6, 0x073e5d47, 0x798b7d4a,
+	0x510da98c, 0x2fb88981, 0x0e7cba8e, 0x70c99a83, 0x584f4e45, 0x26fa6e48,
+	0x0942e7c9, 0x77f7c7c4, 0x5f711302, 0x21c4330f, 0x1cf9751c, 0x624c5511,
+	0x4aca81d7, 0x347fa1da, 0x1bc7285b, 0x65720856, 0x4df4dc90, 0x3341fc9d,
+	0x1285cf92, 0x6c30ef9f, 0x44b63b59, 0x3a031b54, 0x15bb92d5, 0x6b0eb2d8,
+	0x4388661e, 0x3d3d4613, 0x39f2ea38, 0x4747ca35, 0x6fc11ef3, 0x11743efe,
+	0x3eccb77f, 0x40799772, 0x68ff43b4, 0x164a63b9, 0x378e50b6, 0x493b70bb,
+	0x61bda47d, 0x1f088470, 0x30b00df1, 0x4e052dfc, 0x6683f93a, 0x1836d937,
+	0x250b9f24, 0x5bbebf29, 0x73386bef, 0x0d8d4be2, 0x2235c263, 0x5c80e26e,
+	0x740636a8, 0x0ab316a5, 0x2b7725aa, 0x55c205a7, 0x7d44d161, 0x03f1f16c,
+	0x2c4978ed, 0x52fc58e0, 0x7a7a8c26, 0x04cfac2b, 0x73e5d470, 0x0d50f47d,
+	0x25d620bb, 0x5b6300b6, 0x74db8937, 0x0a6ea93a, 0x22e87dfc, 0x5c5d5df1,
+	0x7d996efe, 0x032c4ef3, 0x2baa9a35, 0x551fba38, 0x7aa733b9, 0x041213b4,
+	0x2c94c772, 0x5221e77f, 0x6f1ca16c, 0x11a98161, 0x392f55a7, 0x479a75aa,
+	0x6822fc2b, 0x1697dc26, 0x3e1108e0, 0x40a428ed, 0x61601be2, 0x1fd53bef,
+	0x3753ef29, 0x49e6cf24, 0x665e46a5, 0x18eb66a8, 0x306db26e, 0x4ed89263,
+	0x4a173e48, 0x34a21e45, 0x1c24ca83, 0x6291ea8e, 0x4d29630f, 0x339c4302,
+	0x1b1a97c4, 0x65afb7c9, 0x446b84c6, 0x3adea4cb, 0x1258700d, 0x6ced5000,
+	0x4355d981, 0x3de0f98c, 0x15662d4a, 0x6bd30d47, 0x56ee4b54, 0x285b6b59,
+	0x00ddbf9f, 0x7e689f92, 0x51d01613, 0x2f65361e, 0x07e3e2d8, 0x7956c2d5,
+	0x5892f1da, 0x2627d1d7, 0x0ea10511, 0x7014251c, 0x5facac9d, 0x21198c90,
+	0x099f5856, 0x772a785b, 0x4c921c31, 0x32273c3c, 0x1aa1e8fa, 0x6414c8f7,
+	0x4bac4176, 0x3519617b, 0x1d9fb5bd, 0x632a95b0, 0x42eea6bf, 0x3c5b86b2,
+	0x14dd5274, 0x6a687279, 0x45d0fbf8, 0x3b65dbf5, 0x13e30f33, 0x6d562f3e,
+	0x506b692d, 0x2ede4920, 0x06589de6, 0x78edbdeb, 0x5755346a, 0x29e01467,
+	0x0166c0a1, 0x7fd3e0ac, 0x5e17d3a3, 0x20a2f3ae, 0x08242768, 0x76910765,
+	0x59298ee4, 0x279caee9, 0x0f1a7a2f, 0x71af5a22, 0x7560f609, 0x0bd5d604,
+	0x235302c2, 0x5de622cf, 0x725eab4e, 0x0ceb8b43, 0x246d5f85, 0x5ad87f88,
+	0x7b1c4c87, 0x05a96c8a, 0x2d2fb84c, 0x539a9841, 0x7c2211c0, 0x029731cd,
+	0x2a11e50b, 0x54a4c506, 0x69998315, 0x172ca318, 0x3faa77de, 0x411f57d3,
+	0x6ea7de52, 0x1012fe5f, 0x38942a99, 0x46210a94, 0x67e5399b, 0x19501996,
+	0x31d6cd50, 0x4f63ed5d, 0x60db64dc, 0x1e6e44d1, 0x36e89017, 0x485db01a,
+	0x3f77c841, 0x41c2e84c, 0x69443c8a, 0x17f11c87, 0x38499506, 0x46fcb50b,
+	0x6e7a61cd, 0x10cf41c0, 0x310b72cf, 0x4fbe52c2, 0x67388604, 0x198da609,
+	0x36352f88, 0x48800f85, 0x6006db43, 0x1eb3fb4e, 0x238ebd5d, 0x5d3b9d50,
+	0x75bd4996, 0x0b08699b, 0x24b0e01a, 0x5a05c017, 0x728314d1, 0x0c3634dc,
+	0x2df207d3, 0x534727de, 0x7bc1f318, 0x0574d315, 0x2acc5a94, 0x54797a99,
+	0x7cffae5f, 0x024a8e52, 0x06852279, 0x78300274, 0x50b6d6b2, 0x2e03f6bf,
+	0x01bb7f3e, 0x7f0e5f33, 0x57888bf5, 0x293dabf8, 0x08f998f7, 0x764cb8fa,
+	0x5eca6c3c, 0x207f4c31, 0x0fc7c5b0, 0x7172e5bd, 0x59f4317b, 0x27411176,
+	0x1a7c5765, 0x64c97768, 0x4c4fa3ae, 0x32fa83a3, 0x1d420a22, 0x63f72a2f,
+	0x4b71fee9, 0x35c4dee4, 0x1400edeb, 0x6ab5cde6, 0x42331920, 0x3c86392d,
+	0x133eb0ac, 0x6d8b90a1, 0x450d4467, 0x3bb8646a
+};
 
-struct index {
+struct index_entry {
 	const unsigned char *ptr;
 	unsigned int val;
-	struct index *next;
+	struct index_entry *next;
+};
+
+struct delta_index {
+	const void *src_buf;
+	unsigned long src_size;
+	unsigned int hash_mask;
+	struct index_entry *hash[0];
 };
 
-static struct index ** delta_index(const unsigned char *buf,
-				   unsigned long bufsize,
-				   unsigned long trg_bufsize,
-				   unsigned int *hash_shift)
+struct delta_index * create_delta_index(const void *buf, unsigned long bufsize)
 {
-	unsigned int i, hsize, hshift, hlimit, entries, *hash_count;
-	const unsigned char *data;
-	struct index *entry, **hash;
+	unsigned int i, hsize, hmask, entries, prev_val, *hash_count;
+	const unsigned char *data, *buffer = buf;
+	struct delta_index *index;
+	struct index_entry *entry, **hash;
 	void *mem;
+	unsigned long memsize;
+
+	if (!buf || !bufsize)
+		return NULL;
 
-	/* determine index hash size */
-	entries = bufsize  / BLK_SIZE;
+	/* Determine index hash size.  Note that indexing skips the
+	   first byte to allow for optimizing the rabin polynomial
+	   initialization in create_delta(). */
+	entries = (bufsize - 1)  / RABIN_WINDOW;
 	hsize = entries / 4;
 	for (i = 4; (1 << i) < hsize && i < 31; i++);
 	hsize = 1 << i;
-	hshift = 32 - i;
-	*hash_shift = hshift;
+	hmask = hsize - 1;
 
 	/* allocate lookup index */
-	mem = malloc(hsize * sizeof(*hash) + entries * sizeof(*entry));
+	memsize = sizeof(*index) +
+		  sizeof(*hash) * hsize +
+		  sizeof(*entry) * entries;
+	mem = malloc(memsize);
 	if (!mem)
 		return NULL;
+	index = mem;
+	mem = index + 1;
 	hash = mem;
-	entry = mem + hsize * sizeof(*hash);
+	mem = hash + hsize;
+	entry = mem;
+
+	index->src_buf = buf;
+	index->src_size = bufsize;
+	index->hash_mask = hmask;
 	memset(hash, 0, hsize * sizeof(*hash));
 
 	/* allocate an array to count hash entries */
 	hash_count = calloc(hsize, sizeof(*hash_count));
 	if (!hash_count) {
-		free(hash);
+		free(index);
 		return NULL;
 	}
 
 	/* then populate the index */
-	data = buf + entries * BLK_SIZE - BLK_SIZE;
-	while (data >= buf) {
-		unsigned int val = adler32(0, data, BLK_SIZE);
-		i = HASH(val, hshift);
-		entry->ptr = data;
-		entry->val = val;
-		entry->next = hash[i];
-		hash[i] = entry++;
-		hash_count[i]++;
-		data -= BLK_SIZE;
- 	}
+	prev_val = ~0;
+	for (data = buffer + entries * RABIN_WINDOW - RABIN_WINDOW;
+	     data >= buffer;
+	     data -= RABIN_WINDOW) {
+		unsigned int val = 0;
+		for (i = 1; i <= RABIN_WINDOW; i++)
+			val = ((val << 8) | data[i]) ^ T[val >> RABIN_SHIFT];
+		if (val == prev_val) {
+			/* keep the lowest of consecutive identical blocks */
+			entry[-1].ptr = data + RABIN_WINDOW;
+		} else {
+			prev_val = val;
+			i = val & hmask;
+			entry->ptr = data + RABIN_WINDOW;
+			entry->val = val;
+			entry->next = hash[i];
+			hash[i] = entry++;
+			hash_count[i]++;
+			entries--;
+		}
+	}
 
 	/*
 	 * Determine a limit on the number of entries in the same hash
@@ -91,27 +210,18 @@ static struct index ** delta_index(const unsigned char *buf,
 	 * bucket that would bring us to O(m*n) computing costs (m and n
 	 * corresponding to reference and target buffer sizes).
 	 *
-	 * The more the target buffer is large, the more it is important to
-	 * have small entry lists for each hash buckets.  With such a limit
-	 * the cost is bounded to something more like O(m+n).
-	 */
-	hlimit = (1 << 26) / trg_bufsize;
-	if (hlimit < 4*BLK_SIZE)
-		hlimit = 4*BLK_SIZE;
-
-	/*
-	 * Now make sure none of the hash buckets has more entries than
+	 * Make sure none of the hash buckets has more entries than
 	 * we're willing to test.  Otherwise we cull the entry list
 	 * uniformly to still preserve a good repartition across
 	 * the reference buffer.
 	 */
 	for (i = 0; i < hsize; i++) {
-		if (hash_count[i] < hlimit)
+		if (hash_count[i] < HASH_LIMIT)
 			continue;
 		entry = hash[i];
 		do {
-			struct index *keep = entry;
-			int skip = hash_count[i] / hlimit / 2;
+			struct index_entry *keep = entry;
+			int skip = hash_count[i] / HASH_LIMIT / 2;
 			do {
 				entry = entry->next;
 			} while(--skip && entry);
@@ -120,32 +230,35 @@ static struct index ** delta_index(const unsigned char *buf,
 	}
 	free(hash_count);
 
-	return hash;
+	/* If we didn't use all hash entries, free the unused memory. */
+	if (entries)
+		index = realloc(index, memsize - entries * sizeof(*entry));
+
+	return index;
 }
 
-/* provide the size of the copy opcode given the block offset and size */
-#define COPYOP_SIZE(o, s) \
-    (!!(o & 0xff) + !!(o & 0xff00) + !!(o & 0xff0000) + !!(o & 0xff000000) + \
-     !!(s & 0xff) + !!(s & 0xff00) + 1)
+void free_delta_index(struct delta_index *index)
+{
+	free(index);
+}
 
-/* the maximum size for any opcode */
-#define MAX_OP_SIZE COPYOP_SIZE(0xffffffff, 0xffffffff)
+/*
+ * The maximum size for any opcode sequence, including the initial header
+ * plus rabin window plus biggest copy.
+ */
+#define MAX_OP_SIZE	(5 + 5 + 1 + RABIN_WINDOW + 7)
 
-void *diff_delta(const void *from_buf, unsigned long from_size,
-		 const void *to_buf, unsigned long to_size,
-		 unsigned long *delta_size,
-		 unsigned long max_size)
+void *
+create_delta(const struct delta_index *index,
+	     const void *trg_buf, unsigned long trg_size,
+	     unsigned long *delta_size, unsigned long max_size)
 {
-	unsigned int i, outpos, outsize, hash_shift;
+	unsigned int i, outpos, outsize, val;
 	int inscnt;
 	const unsigned char *ref_data, *ref_top, *data, *top;
 	unsigned char *out;
-	struct index *entry, **hash;
 
-	if (!from_size || !to_size)
-		return NULL;
-	hash = delta_index(from_buf, from_size, to_size, &hash_shift);
-	if (!hash)
+	if (!trg_buf || !trg_size)
 		return NULL;
 
 	outpos = 0;
@@ -153,64 +266,66 @@ void *diff_delta(const void *from_buf, unsigned long from_size,
 	if (max_size && outsize >= max_size)
 		outsize = max_size + MAX_OP_SIZE + 1;
 	out = malloc(outsize);
-	if (!out) {
-		free(hash);
+	if (!out)
 		return NULL;
-	}
-
-	ref_data = from_buf;
-	ref_top = from_buf + from_size;
-	data = to_buf;
-	top = to_buf + to_size;
 
 	/* store reference buffer size */
-	out[outpos++] = from_size;
-	from_size >>= 7;
-	while (from_size) {
-		out[outpos - 1] |= 0x80;
-		out[outpos++] = from_size;
-		from_size >>= 7;
+	i = index->src_size;
+	while (i >= 0x80) {
+		out[outpos++] = i | 0x80;
+		i >>= 7;
 	}
+	out[outpos++] = i;
 
 	/* store target buffer size */
-	out[outpos++] = to_size;
-	to_size >>= 7;
-	while (to_size) {
-		out[outpos - 1] |= 0x80;
-		out[outpos++] = to_size;
-		to_size >>= 7;
+	i = trg_size;
+	while (i >= 0x80) {
+		out[outpos++] = i | 0x80;
+		i >>= 7;
 	}
-
-	inscnt = 0;
+	out[outpos++] = i;
+
+	ref_data = index->src_buf;
+	ref_top = ref_data + index->src_size;
+	data = trg_buf;
+	top = trg_buf + trg_size;
+
+	outpos++;
+	val = 0;
+	for (i = 0; i < RABIN_WINDOW && data < top; i++, data++) {
+		out[outpos++] = *data;
+		val = ((val << 8) | *data) ^ T[val >> RABIN_SHIFT];
+	}
+	inscnt = i;
 
 	while (data < top) {
 		unsigned int moff = 0, msize = 0;
-		if (data + BLK_SIZE <= top) {
-			unsigned int val = adler32(0, data, BLK_SIZE);
-			i = HASH(val, hash_shift);
-			for (entry = hash[i]; entry; entry = entry->next) {
-				const unsigned char *ref = entry->ptr;
-				const unsigned char *src = data;
-				unsigned int ref_size = ref_top - ref;
-				if (entry->val != val)
-					continue;
-				if (ref_size > top - src)
-					ref_size = top - src;
-				if (ref_size > 0x10000)
-					ref_size = 0x10000;
-				if (ref_size <= msize)
-					break;
-				while (ref_size-- && *src++ == *ref)
-					ref++;
-				if (msize < ref - entry->ptr) {
-					/* this is our best match so far */
-					msize = ref - entry->ptr;
-					moff = entry->ptr - ref_data;
-				}
+		struct index_entry *entry;
+		val ^= U[data[-RABIN_WINDOW]];
+		val = ((val << 8) | *data) ^ T[val >> RABIN_SHIFT];
+		i = val & index->hash_mask;
+		for (entry = index->hash[i]; entry; entry = entry->next) {
+			const unsigned char *ref = entry->ptr;
+			const unsigned char *src = data;
+			unsigned int ref_size = ref_top - ref;
+			if (entry->val != val)
+				continue;
+			if (ref_size > top - src)
+				ref_size = top - src;
+			if (ref_size > 0x10000)
+				ref_size = 0x10000;
+			if (ref_size <= msize)
+				break;
+			while (ref_size-- && *src++ == *ref)
+				ref++;
+			if (msize < ref - entry->ptr) {
+				/* this is our best match so far */
+				msize = ref - entry->ptr;
+				moff = entry->ptr - ref_data;
 			}
 		}
 
-		if (!msize || msize < COPYOP_SIZE(moff, msize)) {
+		if (msize < 4) {
 			if (!inscnt)
 				outpos++;
 			out[outpos++] = *data++;
@@ -222,6 +337,20 @@ void *diff_delta(const void *from_buf, unsigned long from_size,
 		} else {
 			unsigned char *op;
 
+			if (msize >= RABIN_WINDOW) {
+				const unsigned char *sk;
+				sk = data + msize - RABIN_WINDOW;
+				val = 0;
+				for (i = 0; i < RABIN_WINDOW; i++)
+					val = ((val << 8) | *sk++) ^ T[val >> RABIN_SHIFT];
+			} else {
+				const unsigned char *sk = data + 1;
+				for (i = 1; i < msize; i++) {
+					val ^= U[sk[-RABIN_WINDOW]];
+					val = ((val << 8) | *sk++) ^ T[val >> RABIN_SHIFT];
+				}
+			}
+
 			if (inscnt) {
 				while (moff && ref_data[moff-1] == data[-1]) {
 					if (msize == 0x10000)
@@ -266,12 +395,10 @@ void *diff_delta(const void *from_buf, unsigned long from_size,
 			if (max_size && outsize >= max_size)
 				outsize = max_size + MAX_OP_SIZE + 1;
 			if (max_size && outpos > max_size)
-				out = NULL;
-			else
-				out = realloc(out, outsize);
+				break;
+			out = realloc(out, outsize);
 			if (!out) {
 				free(tmp);
-				free(hash);
 				return NULL;
 			}
 		}
@@ -280,7 +407,11 @@ void *diff_delta(const void *from_buf, unsigned long from_size,
 	if (inscnt)
 		out[outpos - inscnt - 1] = inscnt;
 
-	free(hash);
+	if (max_size && outpos > max_size) {
+		free(out);
+		return NULL;
+	}
+
 	*delta_size = outpos;
 	return out;
 }
diff --git a/dump-cache-tree.c b/dump-cache-tree.c
new file mode 100644
index 0000000000..1ccaf51773
--- /dev/null
+++ b/dump-cache-tree.c
@@ -0,0 +1,64 @@
+#include "cache.h"
+#include "tree.h"
+#include "cache-tree.h"
+
+
+static void dump_one(struct cache_tree *it, const char *pfx, const char *x)
+{
+	if (it->entry_count < 0)
+		printf("%-40s %s%s (%d subtrees)\n",
+		       "invalid", x, pfx, it->subtree_nr);
+	else
+		printf("%s %s%s (%d entries, %d subtrees)\n",
+		       sha1_to_hex(it->sha1), x, pfx,
+		       it->entry_count, it->subtree_nr);
+}
+
+static int dump_cache_tree(struct cache_tree *it,
+			   struct cache_tree *ref,
+			   const char *pfx)
+{
+	int i;
+	int errs = 0;
+
+	if (!it || !ref)
+		/* missing in either */
+		return 0;
+
+	if (it->entry_count < 0) {
+		dump_one(it, pfx, "");
+		dump_one(ref, pfx, "#(ref) ");
+		if (it->subtree_nr != ref->subtree_nr)
+			errs = 1;
+	}
+	else {
+		dump_one(it, pfx, "");
+		if (memcmp(it->sha1, ref->sha1, 20) ||
+		    ref->entry_count != it->entry_count ||
+		    ref->subtree_nr != it->subtree_nr) {
+			dump_one(ref, pfx, "#(ref) ");
+			errs = 1;
+		}
+	}
+
+	for (i = 0; i < it->subtree_nr; i++) {
+		char path[PATH_MAX];
+		struct cache_tree_sub *down = it->down[i];
+		struct cache_tree_sub *rdwn;
+
+		rdwn = cache_tree_sub(ref, down->name);
+		sprintf(path, "%s%.*s/", pfx, down->namelen, down->name);
+		if (dump_cache_tree(down->cache_tree, rdwn->cache_tree, path))
+			errs = 1;
+	}
+	return errs;
+}
+
+int main(int ac, char **av)
+{
+	struct cache_tree *another = cache_tree();
+	if (read_cache() < 0)
+		die("unable to read index file");
+	cache_tree_update(another, active_cache, active_nr, 0, 1);
+	return dump_cache_tree(active_cache_tree, another, "");
+}
diff --git a/fsck-objects.c b/fsck-objects.c
index 59b25904cb..1922b6d84c 100644
--- a/fsck-objects.c
+++ b/fsck-objects.c
@@ -8,6 +8,7 @@
 #include "tag.h"
 #include "refs.h"
 #include "pack.h"
+#include "cache-tree.h"
 
 #define REACHABLE 0x0001
 
@@ -438,6 +439,28 @@ static int fsck_head_link(void)
 	return 0;
 }
 
+static int fsck_cache_tree(struct cache_tree *it)
+{
+	int i;
+	int err = 0;
+
+	if (0 <= it->entry_count) {
+		struct object *obj = parse_object(it->sha1);
+		if (!obj) {
+			error("%s: invalid sha1 pointer in cache-tree",
+			      sha1_to_hex(it->sha1));
+			return 1;
+		}
+		mark_reachable(obj, REACHABLE);
+		obj->used = 1;
+		if (obj->type != tree_type)
+			err |= objerror(obj, "non-tree in cache-tree");
+	}
+	for (i = 0; i < it->subtree_nr; i++)
+		err |= fsck_cache_tree(it->down[i]->cache_tree);
+	return err;
+}
+
 int main(int argc, char **argv)
 {
 	int i, heads;
@@ -547,6 +570,8 @@ int main(int argc, char **argv)
 			obj->used = 1;
 			mark_reachable(obj, REACHABLE);
 		}
+		if (active_cache_tree)
+			fsck_cache_tree(active_cache_tree);
 	}
 
 	check_connectivity();
diff --git a/git-clone.sh b/git-clone.sh
index 0805168057..227245c865 100755
--- a/git-clone.sh
+++ b/git-clone.sh
@@ -261,11 +261,7 @@ yes,yes)
 	    ;;
 	yes)
 	    mkdir -p "$GIT_DIR/objects/info"
-	    {
-		test -f "$repo/objects/info/alternates" &&
-		cat "$repo/objects/info/alternates";
-		echo "$repo/objects"
-	    } >"$GIT_DIR/objects/info/alternates"
+	    echo "$repo/objects" >> "$GIT_DIR/objects/info/alternates"
 	    ;;
 	esac
 	git-ls-remote "$repo" >"$GIT_DIR/CLONE_HEAD"
diff --git a/git-parse-remote.sh b/git-parse-remote.sh
index c9b899e3d7..187f0883c9 100755
--- a/git-parse-remote.sh
+++ b/git-parse-remote.sh
@@ -10,7 +10,10 @@ get_data_source () {
 		# Not so fast.	This could be the partial URL shorthand...
 		token=$(expr "z$1" : 'z\([^/]*\)/')
 		remainder=$(expr "z$1" : 'z[^/]*/\(.*\)')
-		if test -f "$GIT_DIR/branches/$token"
+		if test "$(git-repo-config --get "remote.$token.url")"
+		then
+			echo config-partial
+		elif test -f "$GIT_DIR/branches/$token"
 		then
 			echo branches-partial
 		else
@@ -18,7 +21,10 @@ get_data_source () {
 		fi
 		;;
 	*)
-		if test -f "$GIT_DIR/remotes/$1"
+		if test "$(git-repo-config --get "remote.$1.url")"
+		then
+			echo config
+		elif test -f "$GIT_DIR/remotes/$1"
 		then
 			echo remotes
 		elif test -f "$GIT_DIR/branches/$1"
@@ -35,6 +41,15 @@ get_remote_url () {
 	case "$data_source" in
 	'')
 		echo "$1" ;;
+	config-partial)
+		token=$(expr "z$1" : 'z\([^/]*\)/')
+		remainder=$(expr "z$1" : 'z[^/]*/\(.*\)')
+		url=$(git-repo-config --get "remote.$token.url")
+		echo "$url/$remainder"
+		;;
+	config)
+		git-repo-config --get "remote.$1.url"
+		;;
 	remotes)
 		sed -ne '/^URL: */{
 			s///p
@@ -56,8 +71,10 @@ get_remote_url () {
 get_remote_default_refs_for_push () {
 	data_source=$(get_data_source "$1")
 	case "$data_source" in
-	'' | branches | branches-partial)
+	'' | config-partial | branches | branches-partial)
 		;; # no default push mapping, just send matching refs.
+	config)
+		git-repo-config --get-all "remote.$1.push" ;;
 	remotes)
 		sed -ne '/^Push: */{
 			s///p
@@ -111,8 +128,11 @@ canon_refs_list_for_fetch () {
 get_remote_default_refs_for_fetch () {
 	data_source=$(get_data_source "$1")
 	case "$data_source" in
-	'' | branches-partial)
+	'' | config-partial | branches-partial)
 		echo "HEAD:" ;;
+	config)
+		canon_refs_list_for_fetch \
+			$(git-repo-config --get-all "remote.$1.fetch") ;;
 	branches)
 		remote_branch=$(sed -ne '/#/s/.*#//p' "$GIT_DIR/branches/$1")
 		case "$remote_branch" in '') remote_branch=master ;; esac
diff --git a/git-repack.sh b/git-repack.sh
index e0c9f323c3..4fb3f26e83 100755
--- a/git-repack.sh
+++ b/git-repack.sh
@@ -48,15 +48,15 @@ name=$(git-rev-list --objects --all $rev_list 2>&1 |
 	exit 1
 if [ -z "$name" ]; then
 	echo Nothing new to pack.
-	exit 0
-fi
-echo "Pack pack-$name created."
+else
+	echo "Pack pack-$name created."
 
-mkdir -p "$PACKDIR" || exit
+	mkdir -p "$PACKDIR" || exit
 
-mv .tmp-pack-$name.pack "$PACKDIR/pack-$name.pack" &&
-mv .tmp-pack-$name.idx  "$PACKDIR/pack-$name.idx" ||
-exit
+	mv .tmp-pack-$name.pack "$PACKDIR/pack-$name.pack" &&
+	mv .tmp-pack-$name.idx  "$PACKDIR/pack-$name.idx" ||
+	exit
+fi
 
 if test "$remove_redundant" = t
 then
diff --git a/git.c b/git.c
index 49ba5189d9..84803a62e6 100644
--- a/git.c
+++ b/git.c
@@ -47,8 +47,10 @@ static void handle_internal_command(int argc, const char **argv, char **envp)
 		{ "whatchanged", cmd_whatchanged },
 		{ "show", cmd_show },
 		{ "push", cmd_push },
+		{ "fmt-patch", cmd_format_patch },
 		{ "count-objects", cmd_count_objects },
 		{ "diff", cmd_diff },
+		{ "grep", cmd_grep },
 	};
 	int i;
 
diff --git a/log-tree.c b/log-tree.c
index b90ba6762a..526d578e98 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -20,6 +20,7 @@ void show_log(struct rev_info *opt, struct log_info *log, const char *sep)
 	int abbrev_commit = opt->abbrev_commit ? opt->abbrev : 40;
 	const char *extra;
 	int len;
+	char* subject = NULL;
 
 	opt->loginfo = NULL;
 	if (!opt->verbose_header) {
@@ -49,19 +50,38 @@ void show_log(struct rev_info *opt, struct log_info *log, const char *sep)
 	/*
 	 * Print header line of header..
 	 */
-	printf("%s%s",
-		opt->commit_format == CMIT_FMT_ONELINE ? "" : "commit ",
-		diff_unique_abbrev(commit->object.sha1, abbrev_commit));
-	if (opt->parents)
-		show_parents(commit, abbrev_commit);
-	if (parent)
-		printf(" (from %s)", diff_unique_abbrev(parent->object.sha1, abbrev_commit));
-	putchar(opt->commit_format == CMIT_FMT_ONELINE ? ' ' : '\n');
+
+	if (opt->commit_format == CMIT_FMT_EMAIL) {
+		if (opt->total > 0) {
+			static char buffer[64];
+			snprintf(buffer, sizeof(buffer),
+					"Subject: [PATCH %d/%d] ",
+					opt->nr, opt->total);
+			subject = buffer;
+		} else if (opt->total == 0)
+			subject = "Subject: [PATCH] ";
+		else
+			subject = "Subject: ";
+
+		printf("From %s  Thu Apr 7 15:13:13 2005\n",
+		       sha1_to_hex(commit->object.sha1));
+	} else {
+		printf("%s%s",
+		       opt->commit_format == CMIT_FMT_ONELINE ? "" : "commit ",
+		       diff_unique_abbrev(commit->object.sha1, abbrev_commit));
+		if (opt->parents)
+			show_parents(commit, abbrev_commit);
+		if (parent) 
+			printf(" (from %s)",
+			       diff_unique_abbrev(parent->object.sha1,
+						  abbrev_commit));
+		putchar(opt->commit_format == CMIT_FMT_ONELINE ? ' ' : '\n');
+	}
 
 	/*
 	 * And then the pretty-printed message itself
 	 */
-	len = pretty_print_commit(opt->commit_format, commit, ~0u, this_header, sizeof(this_header), abbrev);
+	len = pretty_print_commit(opt->commit_format, commit, ~0u, this_header, sizeof(this_header), abbrev, subject);
 	printf("%s%s%s", this_header, extra, sep);
 }
 
@@ -166,15 +186,18 @@ static int log_tree_diff(struct rev_info *opt, struct commit *commit, struct log
 int log_tree_commit(struct rev_info *opt, struct commit *commit)
 {
 	struct log_info log;
+	int shown;
 
 	log.commit = commit;
 	log.parent = NULL;
 	opt->loginfo = &log;
 
-	if (!log_tree_diff(opt, commit, &log) && opt->loginfo && opt->always_show_header) {
+	shown = log_tree_diff(opt, commit, &log);
+	if (!shown && opt->loginfo && opt->always_show_header) {
 		log.parent = NULL;
 		show_log(opt, opt->loginfo, "");
+		shown = 1;
 	}
 	opt->loginfo = NULL;
-	return 0;
+	return shown;
 }
diff --git a/pack-objects.c b/pack-objects.c
index 53caed42dd..523a1c7da8 100644
--- a/pack-objects.c
+++ b/pack-objects.c
@@ -994,6 +994,7 @@ static int type_size_sort(const struct object_entry *a, const struct object_entr
 struct unpacked {
 	struct object_entry *entry;
 	void *data;
+	struct delta_index *index;
 };
 
 /*
@@ -1004,61 +1005,56 @@ struct unpacked {
  * more importantly, the bigger file is likely the more recent
  * one.
  */
-static int try_delta(struct unpacked *cur, struct unpacked *old, unsigned max_depth)
+static int try_delta(struct unpacked *trg, struct unpacked *src,
+		     struct delta_index *src_index, unsigned max_depth)
 {
-	struct object_entry *cur_entry = cur->entry;
-	struct object_entry *old_entry = old->entry;
-	unsigned long size, oldsize, delta_size, sizediff;
-	long max_size;
+	struct object_entry *trg_entry = trg->entry;
+	struct object_entry *src_entry = src->entry;
+	unsigned long size, src_size, delta_size, sizediff, max_size;
 	void *delta_buf;
 
 	/* Don't bother doing diffs between different types */
-	if (cur_entry->type != old_entry->type)
+	if (trg_entry->type != src_entry->type)
 		return -1;
 
 	/* We do not compute delta to *create* objects we are not
 	 * going to pack.
 	 */
-	if (cur_entry->preferred_base)
+	if (trg_entry->preferred_base)
 		return -1;
 
-	/* If the current object is at pack edge, take the depth the
+	/*
+	 * If the current object is at pack edge, take the depth the
 	 * objects that depend on the current object into account --
 	 * otherwise they would become too deep.
 	 */
-	if (cur_entry->delta_child) {
-		if (max_depth <= cur_entry->delta_limit)
+	if (trg_entry->delta_child) {
+		if (max_depth <= trg_entry->delta_limit)
 			return 0;
-		max_depth -= cur_entry->delta_limit;
+		max_depth -= trg_entry->delta_limit;
 	}
-
-	if (old_entry->depth >= max_depth)
+	if (src_entry->depth >= max_depth)
 		return 0;
 
-	/*
-	 * NOTE!
-	 *
-	 * We always delta from the bigger to the smaller, since that's
-	 * more space-efficient (deletes don't have to say _what_ they
-	 * delete).
-	 */
-	size = cur_entry->size;
+	/* Now some size filtering euristics. */
+	size = trg_entry->size;
 	max_size = size / 2 - 20;
-	if (cur_entry->delta)
-		max_size = cur_entry->delta_size-1;
-	oldsize = old_entry->size;
-	sizediff = oldsize < size ? size - oldsize : 0;
+	if (trg_entry->delta)
+		max_size = trg_entry->delta_size-1;
+	src_size = src_entry->size;
+	sizediff = src_size < size ? size - src_size : 0;
 	if (sizediff >= max_size)
 		return 0;
-	delta_buf = diff_delta(old->data, oldsize,
-			       cur->data, size, &delta_size, max_size);
+
+	delta_buf = create_delta(src_index, trg->data, size, &delta_size, max_size);
 	if (!delta_buf)
 		return 0;
-	cur_entry->delta = old_entry;
-	cur_entry->delta_size = delta_size;
-	cur_entry->depth = old_entry->depth + 1;
+
+	trg_entry->delta = src_entry;
+	trg_entry->delta_size = delta_size;
+	trg_entry->depth = src_entry->depth + 1;
 	free(delta_buf);
-	return 0;
+	return 1;
 }
 
 static void progress_interval(int signum)
@@ -1108,12 +1104,17 @@ static void find_deltas(struct object_entry **list, int window, int depth)
 
 		if (entry->size < 50)
 			continue;
-
+		if (n->index)
+			free_delta_index(n->index);
 		free(n->data);
 		n->entry = entry;
 		n->data = read_sha1_file(entry->sha1, type, &size);
 		if (size != entry->size)
-			die("object %s inconsistent object length (%lu vs %lu)", sha1_to_hex(entry->sha1), size, entry->size);
+			die("object %s inconsistent object length (%lu vs %lu)",
+			    sha1_to_hex(entry->sha1), size, entry->size);
+		n->index = create_delta_index(n->data, size);
+		if (!n->index)
+			die("out of memory");
 
 		j = window;
 		while (--j > 0) {
@@ -1124,7 +1125,7 @@ static void find_deltas(struct object_entry **list, int window, int depth)
 			m = array + other_idx;
 			if (!m->entry)
 				break;
-			if (try_delta(n, m, depth) < 0)
+			if (try_delta(n, m, m->index, depth) < 0)
 				break;
 		}
 #if 0
@@ -1144,8 +1145,11 @@ static void find_deltas(struct object_entry **list, int window, int depth)
 	if (progress)
 		fputc('\n', stderr);
 
-	for (i = 0; i < window; ++i)
+	for (i = 0; i < window; ++i) {
+		if (array[i].index)
+			free_delta_index(array[i].index);
 		free(array[i].data);
+	}
 	free(array);
 }
 
diff --git a/patch-delta.c b/patch-delta.c
index affd4f673d..8f318ed8aa 100644
--- a/patch-delta.c
+++ b/patch-delta.c
@@ -13,7 +13,7 @@
 #include <string.h>
 #include "delta.h"
 
-void *patch_delta(void *src_buf, unsigned long src_size,
+void *patch_delta(const void *src_buf, unsigned long src_size,
 		  const void *delta_buf, unsigned long delta_size,
 		  unsigned long *dst_size)
 {
diff --git a/read-cache.c b/read-cache.c
index f97f92d90a..1f71d12578 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -4,11 +4,26 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "cache.h"
+#include "cache-tree.h"
+
+/* Index extensions.
+ *
+ * The first letter should be 'A'..'Z' for extensions that are not
+ * necessary for a correct operation (i.e. optimization data).
+ * When new extensions are added that _needs_ to be understood in
+ * order to correctly interpret the index file, pick character that
+ * is outside the range, to cause the reader to abort.
+ */
+
+#define CACHE_EXT(s) ( (s[0]<<24)|(s[1]<<16)|(s[2]<<8)|(s[3]) )
+#define CACHE_EXT_TREE 0x54524545	/* "TREE" */
 
 struct cache_entry **active_cache = NULL;
 static time_t index_file_timestamp;
 unsigned int active_nr = 0, active_alloc = 0, active_cache_changed = 0;
 
+struct cache_tree *active_cache_tree = NULL;
+
 /*
  * This only updates the "non-critical" parts of the directory
  * cache, ie the parts that aren't tracked by GIT, and only used
@@ -513,6 +528,22 @@ static int verify_hdr(struct cache_header *hdr, unsigned long size)
 	return 0;
 }
 
+static int read_index_extension(const char *ext, void *data, unsigned long sz)
+{
+	switch (CACHE_EXT(ext)) {
+	case CACHE_EXT_TREE:
+		active_cache_tree = cache_tree_read(data, sz);
+		break;
+	default:
+		if (*ext < 'A' || 'Z' < *ext)
+			return error("index uses %.4s extension, which we do not understand",
+				     ext);
+		fprintf(stderr, "ignoring %.4s extension\n", ext);
+		break;
+	}
+	return 0;
+}
+
 int read_cache(void)
 {
 	int fd, i;
@@ -561,6 +592,22 @@ int read_cache(void)
 		active_cache[i] = ce;
 	}
 	index_file_timestamp = st.st_mtime;
+	while (offset <= size - 20 - 8) {
+		/* After an array of active_nr index entries,
+		 * there can be arbitrary number of extended
+		 * sections, each of which is prefixed with
+		 * extension name (4-byte) and section length
+		 * in 4-byte network byte order.
+		 */
+		unsigned long extsize;
+		memcpy(&extsize, map + offset + 4, 4);
+		extsize = ntohl(extsize);
+		if (read_index_extension(map + offset,
+					 map + offset + 8, extsize) < 0)
+			goto unmap;
+		offset += 8;
+		offset += extsize;
+	}
 	return active_nr;
 
 unmap:
@@ -595,6 +642,17 @@ static int ce_write(SHA_CTX *context, int fd, void *data, unsigned int len)
  	return 0;
 }
 
+static int write_index_ext_header(SHA_CTX *context, int fd,
+				  unsigned long ext, unsigned long sz)
+{
+	ext = htonl(ext);
+	sz = htonl(sz);
+	if ((ce_write(context, fd, &ext, 4) < 0) ||
+	    (ce_write(context, fd, &sz, 4) < 0))
+		return -1;
+	return 0;
+}
+
 static int ce_flush(SHA_CTX *context, int fd)
 {
 	unsigned int left = write_buffer_len;
@@ -691,5 +749,19 @@ int write_cache(int newfd, struct cache_entry **cache, int entries)
 		if (ce_write(&c, newfd, ce, ce_size(ce)) < 0)
 			return -1;
 	}
+
+	/* Write extension data here */
+	if (active_cache_tree) {
+		unsigned long sz;
+		void *data = cache_tree_write(active_cache_tree, &sz);
+		if (data &&
+		    !write_index_ext_header(&c, newfd, CACHE_EXT_TREE, sz) &&
+		    !ce_write(&c, newfd, data, sz))
+			;
+		else {
+			free(data);
+			return -1;
+		}
+	}
 	return ce_flush(&c, newfd);
 }
diff --git a/read-tree.c b/read-tree.c
index 26f4f7e323..5d30464106 100644
--- a/read-tree.c
+++ b/read-tree.c
@@ -9,6 +9,7 @@
 
 #include "object.h"
 #include "tree.h"
+#include "cache-tree.h"
 #include <sys/time.h>
 #include <signal.h>
 
@@ -20,6 +21,7 @@ static int trivial_merges_only = 0;
 static int aggressive = 0;
 static int verbose_update = 0;
 static volatile int progress_update = 0;
+static const char *prefix = NULL;
 
 static int head_idx = -1;
 static int merge_size = 0;
@@ -368,7 +370,8 @@ static int unpack_trees(merge_fn_t fn)
 			posns[i] = ((struct tree *) posn->item)->entries;
 			posn = posn->next;
 		}
-		if (unpack_trees_rec(posns, len, "", fn, &indpos))
+		if (unpack_trees_rec(posns, len, prefix ? prefix : "",
+				     fn, &indpos))
 			return -1;
 	}
 
@@ -421,6 +424,12 @@ static void verify_uptodate(struct cache_entry *ce)
 	die("Entry '%s' not uptodate. Cannot merge.", ce->name);
 }
 
+static void invalidate_ce_path(struct cache_entry *ce)
+{
+	if (ce)
+		cache_tree_invalidate_path(active_cache_tree, ce->name);
+}
+
 static int merged_entry(struct cache_entry *merge, struct cache_entry *old)
 {
 	merge->ce_flags |= htons(CE_UPDATE);
@@ -436,8 +445,11 @@ static int merged_entry(struct cache_entry *merge, struct cache_entry *old)
 			*merge = *old;
 		} else {
 			verify_uptodate(old);
+			invalidate_ce_path(old);
 		}
 	}
+	else
+		invalidate_ce_path(merge);
 	merge->ce_flags &= ~htons(CE_STAGEMASK);
 	add_cache_entry(merge, ADD_CACHE_OK_TO_ADD);
 	return 1;
@@ -449,6 +461,7 @@ static int deleted_entry(struct cache_entry *ce, struct cache_entry *old)
 		verify_uptodate(old);
 	ce->ce_mode = 0;
 	add_cache_entry(ce, ADD_CACHE_OK_TO_ADD);
+	invalidate_ce_path(ce);
 	return 1;
 }
 
@@ -668,6 +681,28 @@ static int twoway_merge(struct cache_entry **src)
 		return deleted_entry(oldtree, current);
 }
 
+/*
+ * Bind merge.
+ *
+ * Keep the index entries at stage0, collapse stage1 but make sure
+ * stage0 does not have anything in prefix.
+ */
+static int bind_merge(struct cache_entry **src)
+{
+	struct cache_entry *old = src[0];
+	struct cache_entry *a = src[1];
+
+	if (merge_size != 1)
+		return error("Cannot do a bind merge of %d trees\n",
+			     merge_size);
+	if (!a)
+		return merged_entry(old, NULL);
+	if (old)
+		die("Entry '%s' overlaps.  Cannot bind.", a->name);
+
+	return merged_entry(a, NULL);
+}
+
 /*
  * One-way merge.
  *
@@ -683,8 +718,10 @@ static int oneway_merge(struct cache_entry **src)
 		return error("Cannot do a oneway merge of %d trees",
 			     merge_size);
 
-	if (!a)
+	if (!a) {
+		invalidate_ce_path(old);
 		return 0;
+	}
 	if (old && same(old, a)) {
 		return keep_entry(old);
 	}
@@ -703,6 +740,7 @@ static int read_cache_unmerged(void)
 		struct cache_entry *ce = active_cache[i];
 		if (ce_stage(ce)) {
 			deleted++;
+			invalidate_ce_path(ce);
 			continue;
 		}
 		if (deleted)
@@ -713,7 +751,40 @@ static int read_cache_unmerged(void)
 	return deleted;
 }
 
-static const char read_tree_usage[] = "git-read-tree (<sha> | -m [--aggressive] [-u | -i] <sha1> [<sha2> [<sha3>]])";
+static void prime_cache_tree_rec(struct cache_tree *it, struct tree *tree)
+{
+	struct tree_entry_list *ent;
+	int cnt;
+	
+	memcpy(it->sha1, tree->object.sha1, 20);
+	for (cnt = 0, ent = tree->entries; ent; ent = ent->next) {
+		if (!ent->directory)
+			cnt++;
+		else {
+			struct cache_tree_sub *sub;
+			struct tree *subtree = (struct tree *)ent->item.tree;
+			if (!subtree->object.parsed)
+				parse_tree(subtree);
+			sub = cache_tree_sub(it, ent->name);
+			sub->cache_tree = cache_tree();
+			prime_cache_tree_rec(sub->cache_tree, subtree);
+			cnt += sub->cache_tree->entry_count;
+		}
+	}
+	it->entry_count = cnt;
+}
+
+static void prime_cache_tree(void)
+{
+	struct tree *tree = (struct tree *)trees->item;
+	if (!tree)
+		return;
+	active_cache_tree = cache_tree();
+	prime_cache_tree_rec(active_cache_tree, tree);
+
+}
+
+static const char read_tree_usage[] = "git-read-tree (<sha> | [[-m [--aggressive] | --reset | --prefix=<prefix>] [-u | -i]] <sha1> [<sha2> [<sha3>]])";
 
 static struct cache_file cache_file;
 
@@ -758,9 +829,24 @@ int main(int argc, char **argv)
 			continue;
 		}
 
+		/* "--prefix=<subdirectory>/" means keep the current index
+		 *  entries and put the entries from the tree under the
+		 * given subdirectory.
+		 */
+		if (!strncmp(arg, "--prefix=", 9)) {
+			if (stage || merge || prefix)
+				usage(read_tree_usage);
+			prefix = arg + 9;
+			merge = 1;
+			stage = 1;
+			if (read_cache_unmerged())
+				die("you need to resolve your current index first");
+			continue;
+		}
+
 		/* This differs from "-m" in that we'll silently ignore unmerged entries */
 		if (!strcmp(arg, "--reset")) {
-			if (stage || merge)
+			if (stage || merge || prefix)
 				usage(read_tree_usage);
 			reset = 1;
 			merge = 1;
@@ -781,7 +867,7 @@ int main(int argc, char **argv)
 
 		/* "-m" stands for "merge", meaning we start in stage 1 */
 		if (!strcmp(arg, "-m")) {
-			if (stage || merge)
+			if (stage || merge || prefix)
 				usage(read_tree_usage);
 			if (read_cache_unmerged())
 				die("you need to resolve your current index first");
@@ -803,21 +889,39 @@ int main(int argc, char **argv)
 	if ((update||index_only) && !merge)
 		usage(read_tree_usage);
 
+	if (prefix) {
+		int pfxlen = strlen(prefix);
+		int pos;
+		if (prefix[pfxlen-1] != '/')
+			die("prefix must end with /");
+		if (stage != 2)
+			die("binding merge takes only one tree");
+		pos = cache_name_pos(prefix, pfxlen);
+		if (0 <= pos)
+			die("corrupt index file");
+		pos = -pos-1;
+		if (pos < active_nr &&
+		    !strncmp(active_cache[pos]->name, prefix, pfxlen))
+			die("subdirectory '%s' already exists.", prefix);
+		pos = cache_name_pos(prefix, pfxlen-1);
+		if (0 <= pos)
+			die("file '%.*s' already exists.", pfxlen-1, prefix);
+	}
+
 	if (merge) {
 		if (stage < 2)
 			die("just how do you expect me to merge %d trees?", stage-1);
 		switch (stage - 1) {
 		case 1:
-			fn = oneway_merge;
+			fn = prefix ? bind_merge : oneway_merge;
 			break;
 		case 2:
 			fn = twoway_merge;
 			break;
 		case 3:
-			fn = threeway_merge;
-			break;
 		default:
 			fn = threeway_merge;
+			cache_tree_free(&active_cache_tree);
 			break;
 		}
 
@@ -828,6 +932,18 @@ int main(int argc, char **argv)
 	}
 
 	unpack_trees(fn);
+
+	/*
+	 * When reading only one tree (either the most basic form,
+	 * "-m ent" or "--reset ent" form), we can obtain a fully
+	 * valid cache-tree because the index must match exactly
+	 * what came from the tree.
+	 */
+	if (trees && trees->item && (!merge || (stage == 2))) {
+		cache_tree_free(&active_cache_tree);
+		prime_cache_tree();
+	}
+
 	if (write_cache(newfd, active_cache, active_nr) ||
 	    commit_index_file(&cache_file))
 		die("unable to write new index file");
diff --git a/rev-list.c b/rev-list.c
index 8b0ec388fa..235ae4c7e1 100644
--- a/rev-list.c
+++ b/rev-list.c
@@ -84,7 +84,7 @@ static void show_commit(struct commit *commit)
 		static char pretty_header[16384];
 		pretty_print_commit(revs.commit_format, commit, ~0,
 				    pretty_header, sizeof(pretty_header),
-				    revs.abbrev);
+				    revs.abbrev, NULL);
 		printf("%s%c", pretty_header, hdr_termination);
 	}
 	fflush(stdout);
diff --git a/revision.h b/revision.h
index 48d7b4ca94..62759f7bc0 100644
--- a/revision.h
+++ b/revision.h
@@ -58,6 +58,7 @@ struct rev_info {
 	unsigned int	abbrev;
 	enum cmit_fmt	commit_format;
 	struct log_info *loginfo;
+	int		nr, total;
 
 	/* special limits */
 	int max_count;
diff --git a/sha1_file.c b/sha1_file.c
index 5464828259..b62d0e3dcd 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -217,6 +217,8 @@ char *sha1_pack_index_name(const unsigned char *sha1)
 struct alternate_object_database *alt_odb_list;
 static struct alternate_object_database **alt_odb_tail;
 
+static void read_info_alternates(const char * alternates, int depth);
+
 /*
  * Prepare alternate object database registry.
  *
@@ -232,14 +234,85 @@ static struct alternate_object_database **alt_odb_tail;
  * SHA1, an extra slash for the first level indirection, and the
  * terminating NUL.
  */
-static void link_alt_odb_entries(const char *alt, const char *ep, int sep,
-				 const char *relative_base)
+static int link_alt_odb_entry(const char * entry, int len, const char * relative_base, int depth)
 {
-	const char *cp, *last;
-	struct alternate_object_database *ent;
+	struct stat st;
 	const char *objdir = get_object_directory();
+	struct alternate_object_database *ent;
+	struct alternate_object_database *alt;
+	/* 43 = 40-byte + 2 '/' + terminating NUL */
+	int pfxlen = len;
+	int entlen = pfxlen + 43;
 	int base_len = -1;
 
+	if (*entry != '/' && relative_base) {
+		/* Relative alt-odb */
+		if (base_len < 0)
+			base_len = strlen(relative_base) + 1;
+		entlen += base_len;
+		pfxlen += base_len;
+	}
+	ent = xmalloc(sizeof(*ent) + entlen);
+
+	if (*entry != '/' && relative_base) {
+		memcpy(ent->base, relative_base, base_len - 1);
+		ent->base[base_len - 1] = '/';
+		memcpy(ent->base + base_len, entry, len);
+	}
+	else
+		memcpy(ent->base, entry, pfxlen);
+
+	ent->name = ent->base + pfxlen + 1;
+	ent->base[pfxlen + 3] = '/';
+	ent->base[pfxlen] = ent->base[entlen-1] = 0;
+
+	/* Detect cases where alternate disappeared */
+	if (stat(ent->base, &st) || !S_ISDIR(st.st_mode)) {
+		error("object directory %s does not exist; "
+		      "check .git/objects/info/alternates.",
+		      ent->base);
+		free(ent);
+		return -1;
+	}
+
+	/* Prevent the common mistake of listing the same
+	 * thing twice, or object directory itself.
+	 */
+	for (alt = alt_odb_list; alt; alt = alt->next) {
+		if (!memcmp(ent->base, alt->base, pfxlen)) {
+			free(ent);
+			return -1;
+		}
+	}
+	if (!memcmp(ent->base, objdir, pfxlen)) {
+		free(ent);
+		return -1;
+	}
+
+	/* add the alternate entry */
+	*alt_odb_tail = ent;
+	alt_odb_tail = &(ent->next);
+	ent->next = NULL;
+
+	/* recursively add alternates */
+	read_info_alternates(ent->base, depth + 1);
+
+	ent->base[pfxlen] = '/';
+
+	return 0;
+}
+
+static void link_alt_odb_entries(const char *alt, const char *ep, int sep,
+				 const char *relative_base, int depth)
+{
+	const char *cp, *last;
+
+	if (depth > 5) {
+		error("%s: ignoring alternate object stores, nesting too deep.",
+				relative_base);
+		return;
+	}
+
 	last = alt;
 	while (last < ep) {
 		cp = last;
@@ -249,60 +322,15 @@ static void link_alt_odb_entries(const char *alt, const char *ep, int sep,
 			last = cp + 1;
 			continue;
 		}
-		for ( ; cp < ep && *cp != sep; cp++)
-			;
+		while (cp < ep && *cp != sep)
+			cp++;
 		if (last != cp) {
-			struct stat st;
-			struct alternate_object_database *alt;
-			/* 43 = 40-byte + 2 '/' + terminating NUL */
-			int pfxlen = cp - last;
-			int entlen = pfxlen + 43;
-
-			if (*last != '/' && relative_base) {
-				/* Relative alt-odb */
-				if (base_len < 0)
-					base_len = strlen(relative_base) + 1;
-				entlen += base_len;
-				pfxlen += base_len;
-			}
-			ent = xmalloc(sizeof(*ent) + entlen);
-
-			if (*last != '/' && relative_base) {
-				memcpy(ent->base, relative_base, base_len - 1);
-				ent->base[base_len - 1] = '/';
-				memcpy(ent->base + base_len,
-				       last, cp - last);
-			}
-			else
-				memcpy(ent->base, last, pfxlen);
-
-			ent->name = ent->base + pfxlen + 1;
-			ent->base[pfxlen + 3] = '/';
-			ent->base[pfxlen] = ent->base[entlen-1] = 0;
-
-			/* Detect cases where alternate disappeared */
-			if (stat(ent->base, &st) || !S_ISDIR(st.st_mode)) {
-				error("object directory %s does not exist; "
-				      "check .git/objects/info/alternates.",
-				      ent->base);
-				goto bad;
-			}
-			ent->base[pfxlen] = '/';
-
-			/* Prevent the common mistake of listing the same
-			 * thing twice, or object directory itself.
-			 */
-			for (alt = alt_odb_list; alt; alt = alt->next)
-				if (!memcmp(ent->base, alt->base, pfxlen))
-					goto bad;
-			if (!memcmp(ent->base, objdir, pfxlen)) {
-			bad:
-				free(ent);
-			}
-			else {
-				*alt_odb_tail = ent;
-				alt_odb_tail = &(ent->next);
-				ent->next = NULL;
+			if ((*last != '/') && depth) {
+				error("%s: ignoring relative alternate object store %s",
+						relative_base, last);
+			} else {
+				link_alt_odb_entry(last, cp - last,
+						relative_base, depth);
 			}
 		}
 		while (cp < ep && *cp == sep)
@@ -311,23 +339,14 @@ static void link_alt_odb_entries(const char *alt, const char *ep, int sep,
 	}
 }
 
-void prepare_alt_odb(void)
+static void read_info_alternates(const char * relative_base, int depth)
 {
-	char path[PATH_MAX];
 	char *map;
-	int fd;
 	struct stat st;
-	char *alt;
-
-	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
-	if (!alt) alt = "";
-
-	if (alt_odb_tail)
-		return;
-	alt_odb_tail = &alt_odb_list;
-	link_alt_odb_entries(alt, alt + strlen(alt), ':', NULL);
+	char path[PATH_MAX];
+	int fd;
 
-	sprintf(path, "%s/info/alternates", get_object_directory());
+	sprintf(path, "%s/info/alternates", relative_base);
 	fd = open(path, O_RDONLY);
 	if (fd < 0)
 		return;
@@ -340,11 +359,26 @@ void prepare_alt_odb(void)
 	if (map == MAP_FAILED)
 		return;
 
-	link_alt_odb_entries(map, map + st.st_size, '\n',
-			     get_object_directory());
+	link_alt_odb_entries(map, map + st.st_size, '\n', relative_base, depth);
+
 	munmap(map, st.st_size);
 }
 
+void prepare_alt_odb(void)
+{
+	char *alt;
+
+	alt = getenv(ALTERNATE_DB_ENVIRONMENT);
+	if (!alt) alt = "";
+
+	if (alt_odb_tail)
+		return;
+	alt_odb_tail = &alt_odb_list;
+	link_alt_odb_entries(alt, alt + strlen(alt), ':', NULL, 0);
+
+	read_info_alternates(get_object_directory(), 0);
+}
+
 static char *find_sha1_file(const unsigned char *sha1, struct stat *st)
 {
 	char *name = sha1_file_name(sha1);
diff --git a/sha1_name.c b/sha1_name.c
index 345935bb2b..dc6835520c 100644
--- a/sha1_name.c
+++ b/sha1_name.c
@@ -458,17 +458,56 @@ int get_sha1(const char *name, unsigned char *sha1)
 {
 	int ret;
 	unsigned unused;
+	int namelen = strlen(name);
+	const char *cp;
 
 	prepare_alt_odb();
-	ret = get_sha1_1(name, strlen(name), sha1);
-	if (ret < 0) {
-		const char *cp = strchr(name, ':');
-		if (cp) {
-			unsigned char tree_sha1[20];
-			if (!get_sha1_1(name, cp-name, tree_sha1))
-				return get_tree_entry(tree_sha1, cp+1, sha1,
-						      &unused);
+	ret = get_sha1_1(name, namelen, sha1);
+	if (!ret)
+		return ret;
+	/* sha1:path --> object name of path in ent sha1
+	 * :path -> object name of path in index
+	 * :[0-3]:path -> object name of path in index at stage
+	 */
+	if (name[0] == ':') {
+		int stage = 0;
+		struct cache_entry *ce;
+		int pos;
+		if (namelen < 3 ||
+		    name[2] != ':' ||
+		    name[1] < '0' || '3' < name[1])
+			cp = name + 1;
+		else {
+			stage = name[1] - '0';
+			cp = name + 3;
 		}
+		namelen = namelen - (cp - name);
+		if (!active_cache)
+			read_cache();
+		if (active_nr < 0)
+			return -1;
+		pos = cache_name_pos(cp, namelen);
+		if (pos < 0)
+			pos = -pos - 1;
+		while (pos < active_nr) {
+			ce = active_cache[pos];
+			if (ce_namelen(ce) != namelen ||
+			    memcmp(ce->name, cp, namelen))
+				break;
+			if (ce_stage(ce) == stage) {
+				memcpy(sha1, ce->sha1, 20);
+				return 0;
+			}
+			pos++;
+		}
+		return -1;
+	}
+	cp = strchr(name, ':');
+	if (cp) {
+		unsigned char tree_sha1[20];
+		if (!get_sha1_1(name, cp-name, tree_sha1))
+			return get_tree_entry(tree_sha1, cp+1, sha1,
+					      &unused);
 	}
 	return ret;
 }
diff --git a/show-branch.c b/show-branch.c
index 268c57b180..bbe26c2e7a 100644
--- a/show-branch.c
+++ b/show-branch.c
@@ -259,7 +259,7 @@ static void show_one_commit(struct commit *commit, int no_name)
 	struct commit_name *name = commit->object.util;
 	if (commit->object.parsed)
 		pretty_print_commit(CMIT_FMT_ONELINE, commit, ~0,
-				    pretty, sizeof(pretty), 0);
+				    pretty, sizeof(pretty), 0, NULL);
 	else
 		strcpy(pretty, "(unavailable)");
 	if (!strncmp(pretty, "[PATCH] ", 8))
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index cf33989b56..2c9bbb59b0 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -195,6 +195,20 @@ test_expect_success \
     'git-ls-tree -r output for a known tree.' \
     'diff current expected'
 
+test_expect_success \
+    'writing partial tree out with git-write-tree --prefix.' \
+    'ptree=$(git-write-tree --prefix=path3)'
+test_expect_success \
+    'validate object ID for a known tree.' \
+    'test "$ptree" = 21ae8269cacbe57ae09138dcc3a2887f904d02b3'
+
+test_expect_success \
+    'writing partial tree out with git-write-tree --prefix.' \
+    'ptree=$(git-write-tree --prefix=path3/subp3)'
+test_expect_success \
+    'validate object ID for a known tree.' \
+    'test "$ptree" = 3c5e5399f3a333eddecce7a9b9465b63f65f51e2'
+
 ################################################################
 rm .git/index
 test_expect_success \
diff --git a/t/t1300-repo-config.sh b/t/t1300-repo-config.sh
index 1bf728fb06..7090ea92c1 100755
--- a/t/t1300-repo-config.sh
+++ b/t/t1300-repo-config.sh
@@ -278,5 +278,36 @@ git-repo-config > output 2>&1
 test_expect_success 'no arguments, but no crash' \
 	"test $? = 129 && grep usage output"
 
+cat > .git/config << EOF
+[a.b]
+	c = d
+EOF
+
+git-repo-config a.x y
+
+cat > expect << EOF
+[a.b]
+	c = d
+[a]
+	x = y
+EOF
+
+test_expect_success 'new section is partial match of another' 'cmp .git/config expect'
+
+git-repo-config b.x y
+git-repo-config a.b c
+
+cat > expect << EOF
+[a.b]
+	c = d
+[a]
+	x = y
+	b = c
+[b]
+	x = y
+EOF
+
+test_expect_success 'new variable inserts into proper section' 'cmp .git/config expect'
+
 test_done
 
diff --git a/t/t2101-update-index-reupdate.sh b/t/t2101-update-index-reupdate.sh
new file mode 100755
index 0000000000..77aed8d800
--- /dev/null
+++ b/t/t2101-update-index-reupdate.sh
@@ -0,0 +1,82 @@
+#!/bin/sh
+#
+# Copyright (c) 2006 Junio C Hamano
+#
+
+test_description='git-update-index --again test.
+'
+
+. ./test-lib.sh
+
+test_expect_success 'update-index --add' \
+	'echo hello world >file1 &&
+	 echo goodbye people >file2 &&
+	 git-update-index --add file1 file2 &&
+	 git-ls-files -s >current &&
+	 cmp current - <<\EOF
+100644 3b18e512dba79e4c8300dd08aeb37f8e728b8dad 0	file1
+100644 9db8893856a8a02eaa73470054b7c1c5a7c82e47 0	file2
+EOF'
+
+test_expect_success 'update-index --again' \
+	'rm -f file1 &&
+	echo hello everybody >file2 &&
+	if git-update-index --again
+	then
+		echo should have refused to remove file1
+		exit 1
+	else
+		echo happy - failed as expected
+	fi &&
+	 git-ls-files -s >current &&
+	 cmp current - <<\EOF
+100644 3b18e512dba79e4c8300dd08aeb37f8e728b8dad 0	file1
+100644 9db8893856a8a02eaa73470054b7c1c5a7c82e47 0	file2
+EOF'
+
+test_expect_success 'update-index --remove --again' \
+	'git-update-index --remove --again &&
+	 git-ls-files -s >current &&
+	 cmp current - <<\EOF
+100644 0f1ae1422c2bf43f117d3dbd715c988a9ed2103f 0	file2
+EOF'
+
+test_expect_success 'first commit' 'git-commit -m initial'
+
+test_expect_success 'update-index again' \
+	'mkdir -p dir1 &&
+	echo hello world >dir1/file3 &&
+	echo goodbye people >file2 &&
+	git-update-index --add file2 dir1/file3 &&
+	echo hello everybody >file2
+	echo happy >dir1/file3 &&
+	git-update-index --again &&
+	git-ls-files -s >current &&
+	cmp current - <<\EOF
+100644 53ab446c3f4e42ce9bb728a0ccb283a101be4979 0	dir1/file3
+100644 0f1ae1422c2bf43f117d3dbd715c988a9ed2103f 0	file2
+EOF'
+
+test_expect_success 'update-index --update from subdir' \
+	'echo not so happy >file2 &&
+	cd dir1 &&
+	cat ../file2 >file3 &&
+	git-update-index --again &&
+	cd .. &&
+	git-ls-files -s >current &&
+	cmp current - <<\EOF
+100644 d7fb3f695f06c759dbf3ab00046e7cc2da22d10f 0	dir1/file3
+100644 0f1ae1422c2bf43f117d3dbd715c988a9ed2103f 0	file2
+EOF'
+
+test_expect_success 'update-index --update with pathspec' \
+	'echo very happy >file2 &&
+	cat file2 >dir1/file3 &&
+	git-update-index --again dir1/ &&
+	git-ls-files -s >current &&
+	cmp current - <<\EOF
+100644 594fb5bb1759d90998e2bf2a38261ae8e243c760 0	dir1/file3
+100644 0f1ae1422c2bf43f117d3dbd715c988a9ed2103f 0	file2
+EOF'
+
+test_done
diff --git a/t/t5700-clone-reference.sh b/t/t5700-clone-reference.sh
new file mode 100755
index 0000000000..916ee15ba1
--- /dev/null
+++ b/t/t5700-clone-reference.sh
@@ -0,0 +1,78 @@
+#!/bin/sh
+#
+# Copyright (C) 2006 Martin Waitz <tali@admingilde.org>
+#
+
+test_description='test clone --reference'
+. ./test-lib.sh
+
+base_dir=`pwd`
+
+test_expect_success 'preparing first repository' \
+'test_create_repo A && cd A &&
+echo first > file1 &&
+git add file1 &&
+git commit -m initial'
+
+cd "$base_dir"
+
+test_expect_success 'preparing second repository' \
+'git clone A B && cd B &&
+echo second > file2 &&
+git add file2 &&
+git commit -m addition &&
+git repack -a -d &&
+git prune'
+
+cd "$base_dir"
+
+test_expect_success 'cloning with reference' \
+'git clone -l -s --reference B A C'
+
+cd "$base_dir"
+
+test_expect_success 'existance of info/alternates' \
+'test `wc -l <C/.git/objects/info/alternates` = 2'
+
+cd "$base_dir"
+
+test_expect_success 'pulling from reference' \
+'cd C &&
+git pull ../B'
+
+cd "$base_dir"
+
+test_expect_success 'that reference gets used' \
+'cd C &&
+echo "0 objects, 0 kilobytes" > expected &&
+git count-objects > current &&
+diff expected current'
+
+cd "$base_dir"
+
+test_expect_success 'updating origin' \
+'cd A &&
+echo third > file3 &&
+git add file3 &&
+git commit -m update &&
+git repack -a -d &&
+git prune'
+
+cd "$base_dir"
+
+test_expect_success 'pulling changes from origin' \
+'cd C &&
+git pull origin'
+
+cd "$base_dir"
+
+# the 2 local objects are commit and tree from the merge
+test_expect_success 'that alternate to origin gets used' \
+'cd C &&
+echo "2 objects" > expected &&
+git count-objects | cut -d, -f1 > current &&
+diff expected current'
+
+cd "$base_dir"
+
+test_done
diff --git a/t/t5710-info-alternate.sh b/t/t5710-info-alternate.sh
new file mode 100755
index 0000000000..097d037f5d
--- /dev/null
+++ b/t/t5710-info-alternate.sh
@@ -0,0 +1,105 @@
+#!/bin/sh
+#
+# Copyright (C) 2006 Martin Waitz <tali@admingilde.org>
+#
+
+test_description='test transitive info/alternate entries'
+. ./test-lib.sh
+
+# test that a file is not reachable in the current repository
+# but that it is after creating a info/alternate entry
+reachable_via() {
+	alternate="$1"
+	file="$2"
+	if git cat-file -e "HEAD:$file"; then return 1; fi
+	echo "$alternate" >> .git/objects/info/alternate
+	git cat-file -e "HEAD:$file"
+}
+
+test_valid_repo() {
+	git fsck-objects --full > fsck.log &&
+	test `wc -l < fsck.log` = 0
+}
+
+base_dir=`pwd`
+
+test_expect_success 'preparing first repository' \
+'test_create_repo A && cd A &&
+echo "Hello World" > file1 &&
+git add file1 &&
+git commit -m "Initial commit" file1 &&
+git repack -a -d &&
+git prune'
+
+cd "$base_dir"
+
+test_expect_success 'preparing second repository' \
+'git clone -l -s A B && cd B &&
+echo "foo bar" > file2 &&
+git add file2 &&
+git commit -m "next commit" file2 &&
+git repack -a -d -l &&
+git prune'
+
+cd "$base_dir"
+
+test_expect_success 'preparing third repository' \
+'git clone -l -s B C && cd C &&
+echo "Goodbye, cruel world" > file3 &&
+git add file3 &&
+git commit -m "one more" file3 &&
+git repack -a -d -l &&
+git prune'
+
+cd "$base_dir"
+
+test_expect_failure 'creating too deep nesting' \
+'git clone -l -s C D &&
+git clone -l -s D E &&
+git clone -l -s E F &&
+git clone -l -s F G &&
+test_valid_repo'
+
+cd "$base_dir"
+
+test_expect_success 'validity of third repository' \
+'cd C &&
+test_valid_repo'
+
+cd "$base_dir"
+
+test_expect_success 'validity of fourth repository' \
+'cd D &&
+test_valid_repo'
+
+cd "$base_dir"
+
+test_expect_success 'breaking of loops' \
+"echo '$base_dir/B/.git/objects' >> '$base_dir'/A/.git/objects/info/alternates&&
+cd C &&
+test_valid_repo"
+
+cd "$base_dir"
+
+test_expect_failure 'that info/alternates is neccessary' \
+'cd C &&
+rm .git/objects/info/alternates &&
+test_valid_repo'
+
+cd "$base_dir"
+
+test_expect_success 'that relative alternate is possible for current dir' \
+'cd C &&
+echo "../../../B/.git/objects" > .git/objects/info/alternates &&
+test_valid_repo'
+
+cd "$base_dir"
+
+test_expect_failure 'that relative alternate is only possible for current dir' \
+'cd D &&
+test_valid_repo'
+
+cd "$base_dir"
+
+test_done
+
diff --git a/update-index.c b/update-index.c
index 9fa3d2bb96..f6b09a4800 100644
--- a/update-index.c
+++ b/update-index.c
@@ -6,6 +6,7 @@
 #include "cache.h"
 #include "strbuf.h"
 #include "quote.h"
+#include "cache-tree.h"
 #include "tree-walk.h"
 
 /*
@@ -71,6 +72,7 @@ static int mark_valid(const char *path)
 			active_cache[pos]->ce_flags &= ~htons(CE_VALID);
 			break;
 		}
+		cache_tree_invalidate_path(active_cache_tree, path);
 		active_cache_changed = 1;
 		return 0;
 	}
@@ -84,6 +86,12 @@ static int add_file_to_cache(const char *path)
 	struct stat st;
 
 	status = lstat(path, &st);
+
+	/* We probably want to do this in remove_file_from_cache() and
+	 * add_cache_entry() instead...
+	 */
+	cache_tree_invalidate_path(active_cache_tree, path);
+
 	if (status < 0 || S_ISDIR(st.st_mode)) {
 		/* When we used to have "path" and now we want to add
 		 * "path/file", we need a way to remove "path" before
@@ -326,6 +334,7 @@ static int add_cacheinfo(unsigned int mode, const unsigned char *sha1,
 		return error("%s: cannot add to the index - missing --add option?",
 			     path);
 	report("add '%s'", path);
+	cache_tree_invalidate_path(active_cache_tree, path);
 	return 0;
 }
 
@@ -350,6 +359,7 @@ static void chmod_path(int flip, const char *path)
 	default:
 		goto fail;
 	}
+	cache_tree_invalidate_path(active_cache_tree, path);
 	active_cache_changed = 1;
 	report("chmod %cx '%s'", flip, path);
 	return;
@@ -364,23 +374,27 @@ static void update_one(const char *path, const char *prefix, int prefix_length)
 	const char *p = prefix_path(prefix, prefix_length, path);
 	if (!verify_path(p)) {
 		fprintf(stderr, "Ignoring path %s\n", path);
-		return;
+		goto free_return;
 	}
 	if (mark_valid_only) {
 		if (mark_valid(p))
 			die("Unable to mark file %s", path);
-		return;
+		goto free_return;
 	}
+	cache_tree_invalidate_path(active_cache_tree, path);
 
 	if (force_remove) {
 		if (remove_file_from_cache(p))
 			die("git-update-index: unable to remove %s", path);
 		report("remove '%s'", path);
-		return;
+		goto free_return;
 	}
 	if (add_file_to_cache(p))
 		die("Unable to process file %s", path);
 	report("add '%s'", path);
+ free_return:
+	if (p < path || p > path + strlen(path))
+		free((char*)p);
 }
 
 static void read_index_info(int line_termination)
@@ -446,6 +460,7 @@ static void read_index_info(int line_termination)
 				free(path_name);
 			continue;
 		}
+		cache_tree_invalidate_path(active_cache_tree, path_name);
 
 		if (!mode) {
 			/* mode == 0 means there is no such path -- remove */
@@ -473,7 +488,7 @@ static void read_index_info(int line_termination)
 }
 
 static const char update_index_usage[] =
-"git-update-index [-q] [--add] [--replace] [--remove] [--unmerged] [--refresh] [--really-refresh] [--cacheinfo] [--chmod=(+|-)x] [--assume-unchanged] [--info-only] [--force-remove] [--stdin] [--index-info] [--unresolve] [--ignore-missing] [-z] [--verbose] [--] <file>...";
+"git-update-index [-q] [--add] [--replace] [--remove] [--unmerged] [--refresh] [--really-refresh] [--cacheinfo] [--chmod=(+|-)x] [--assume-unchanged] [--info-only] [--force-remove] [--stdin] [--index-info] [--unresolve] [--again] [--ignore-missing] [-z] [--verbose] [--] <file>...";
 
 static unsigned char head_sha1[20];
 static unsigned char merge_head_sha1[20];
@@ -488,11 +503,13 @@ static struct cache_entry *read_one_ent(const char *which,
 	struct cache_entry *ce;
 
 	if (get_tree_entry(ent, path, sha1, &mode)) {
-		error("%s: not in %s branch.", path, which);
+		if (which)
+			error("%s: not in %s branch.", path, which);
 		return NULL;
 	}
 	if (mode == S_IFDIR) {
-		error("%s: not a blob in %s branch.", path, which);
+		if (which)
+			error("%s: not a blob in %s branch.", path, which);
 		return NULL;
 	}
 	size = cache_entry_size(namelen);
@@ -550,6 +567,7 @@ static int unresolve_one(const char *path)
 		goto free_return;
 	}
 
+	cache_tree_invalidate_path(active_cache_tree, path);
 	remove_file_from_cache(path);
 	if (add_cache_entry(ce_2, ADD_CACHE_OK_TO_ADD)) {
 		error("%s: cannot add our version to the index.", path);
@@ -576,7 +594,8 @@ static void read_head_pointers(void)
 	}
 }
 
-static int do_unresolve(int ac, const char **av)
+static int do_unresolve(int ac, const char **av,
+			const char *prefix, int prefix_length)
 {
 	int i;
 	int err = 0;
@@ -588,11 +607,57 @@ static int do_unresolve(int ac, const char **av)
 
 	for (i = 1; i < ac; i++) {
 		const char *arg = av[i];
-		err |= unresolve_one(arg);
+		const char *p = prefix_path(prefix, prefix_length, arg);
+		err |= unresolve_one(p);
+		if (p < arg || p > arg + strlen(arg))
+			free((char*)p);
 	}
 	return err;
 }
 
+static int do_reupdate(int ac, const char **av,
+		       const char *prefix, int prefix_length)
+{
+	/* Read HEAD and run update-index on paths that are
+	 * merged and already different between index and HEAD.
+	 */
+	int pos;
+	int has_head = 1;
+	const char **pathspec = get_pathspec(prefix, av + 1);
+
+	if (read_ref(git_path("HEAD"), head_sha1))
+		/* If there is no HEAD, that means it is an initial
+		 * commit.  Update everything in the index.
+		 */
+		has_head = 0;
+ redo:
+	for (pos = 0; pos < active_nr; pos++) {
+		struct cache_entry *ce = active_cache[pos];
+		struct cache_entry *old = NULL;
+		int save_nr;
+
+		if (ce_stage(ce) || !ce_path_match(ce, pathspec))
+			continue;
+		if (has_head)
+			old = read_one_ent(NULL, head_sha1,
+					   ce->name, ce_namelen(ce), 0);
+		if (old && ce->ce_mode == old->ce_mode &&
+		    !memcmp(ce->sha1, old->sha1, 20)) {
+			free(old);
+			continue; /* unchanged */
+		}
+		/* Be careful.  The working tree may not have the
+		 * path anymore, in which case, under 'allow_remove',
+		 * or worse yet 'allow_replace', active_nr may decrease.
+		 */
+		save_nr = active_nr;
+		update_one(ce->name + prefix_length, prefix, prefix_length);
+		if (save_nr != active_nr)
+			goto redo;
+	}
+	return 0;
+}
+
 int main(int argc, const char **argv)
 {
 	int i, newfd, entries, has_errors = 0, line_termination = '\n';
@@ -704,7 +769,15 @@ int main(int argc, const char **argv)
 				break;
 			}
 			if (!strcmp(path, "--unresolve")) {
-				has_errors = do_unresolve(argc - i, argv + i);
+				has_errors = do_unresolve(argc - i, argv + i,
+							  prefix, prefix_length);
+				if (has_errors)
+					active_cache_changed = 0;
+				goto finish;
+			}
+			if (!strcmp(path, "--again")) {
+				has_errors = do_reupdate(argc - i, argv + i,
+							 prefix, prefix_length);
 				if (has_errors)
 					active_cache_changed = 0;
 				goto finish;
@@ -730,6 +803,7 @@ int main(int argc, const char **argv)
 		strbuf_init(&buf);
 		while (1) {
 			char *path_name;
+			const char *p;
 			read_line(&buf, stdin, line_termination);
 			if (buf.eof)
 				break;
@@ -737,11 +811,12 @@ int main(int argc, const char **argv)
 				path_name = unquote_c_style(buf.buf, NULL);
 			else
 				path_name = buf.buf;
-			update_one(path_name, prefix, prefix_length);
-			if (set_executable_bit) {
-				const char *p = prefix_path(prefix, prefix_length, path_name);
+			p = prefix_path(prefix, prefix_length, path_name);
+			update_one(p, NULL, 0);
+			if (set_executable_bit)
 				chmod_path(set_executable_bit, p);
-			}
+			if (p < path_name || p > path_name + strlen(path_name))
+				free((char*) p);
 			if (path_name != buf.buf)
 				free(path_name);
 		}
diff --git a/write-tree.c b/write-tree.c
index dcad6e6670..895e7a359d 100644
--- a/write-tree.c
+++ b/write-tree.c
@@ -5,154 +5,68 @@
  */
 #include "cache.h"
 #include "tree.h"
+#include "cache-tree.h"
 
 static int missing_ok = 0;
+static char *prefix = NULL;
 
-static int check_valid_sha1(unsigned char *sha1)
-{
-	int ret;
-
-	/* If we were anal, we'd check that the sha1 of the contents actually matches */
-	ret = has_sha1_file(sha1);
-	if (ret == 0)
-		perror(sha1_file_name(sha1));
-	return ret ? 0 : -1;
-}
-
-static int write_tree(struct cache_entry **cachep, int maxentries, const char *base, int baselen, unsigned char *returnsha1)
-{
-	unsigned char subdir_sha1[20];
-	unsigned long size, offset;
-	char *buffer;
-	int nr;
-
-	/* Guess at some random initial size */
-	size = 8192;
-	buffer = xmalloc(size);
-	offset = 0;
-
-	nr = 0;
-	while (nr < maxentries) {
-		struct cache_entry *ce = cachep[nr];
-		const char *pathname = ce->name, *filename, *dirname;
-		int pathlen = ce_namelen(ce), entrylen;
-		unsigned char *sha1;
-		unsigned int mode;
-
-		/* Did we hit the end of the directory? Return how many we wrote */
-		if (baselen >= pathlen || memcmp(base, pathname, baselen))
-			break;
-
-		sha1 = ce->sha1;
-		mode = ntohl(ce->ce_mode);
-
-		/* Do we have _further_ subdirectories? */
-		filename = pathname + baselen;
-		dirname = strchr(filename, '/');
-		if (dirname) {
-			int subdir_written;
-
-			subdir_written = write_tree(cachep + nr, maxentries - nr, pathname, dirname-pathname+1, subdir_sha1);
-			nr += subdir_written;
-
-			/* Now we need to write out the directory entry into this tree.. */
-			mode = S_IFDIR;
-			pathlen = dirname - pathname;
-
-			/* ..but the directory entry doesn't count towards the total count */
-			nr--;
-			sha1 = subdir_sha1;
-		}
+static const char write_tree_usage[] =
+"git-write-tree [--missing-ok] [--prefix=<prefix>/]";
 
-		if (!missing_ok && check_valid_sha1(sha1) < 0)
-			exit(1);
-
-		entrylen = pathlen - baselen;
-		if (offset + entrylen + 100 > size) {
-			size = alloc_nr(offset + entrylen + 100);
-			buffer = xrealloc(buffer, size);
-		}
-		offset += sprintf(buffer + offset, "%o %.*s", mode, entrylen, filename);
-		buffer[offset++] = 0;
-		memcpy(buffer + offset, sha1, 20);
-		offset += 20;
-		nr++;
-	}
-
-	write_sha1_file(buffer, offset, tree_type, returnsha1);
-	free(buffer);
-	return nr;
-}
-
-static const char write_tree_usage[] = "git-write-tree [--missing-ok]";
+static struct cache_file cache_file;
 
 int main(int argc, char **argv)
 {
-	int i, funny;
-	int entries;
-	unsigned char sha1[20];
-	
+	int entries, was_valid, newfd;
+
 	setup_git_directory();
 
+	newfd = hold_index_file_for_update(&cache_file, get_index_file());
 	entries = read_cache();
-	if (argc == 2) {
-		if (!strcmp(argv[1], "--missing-ok"))
+
+	while (1 < argc) {
+		char *arg = argv[1];
+		if (!strcmp(arg, "--missing-ok"))
 			missing_ok = 1;
+		else if (!strncmp(arg, "--prefix=", 9))
+			prefix = arg + 9;
 		else
 			die(write_tree_usage);
+		argc--; argv++;
 	}
-	
+
 	if (argc > 2)
 		die("too many options");
 
 	if (entries < 0)
 		die("git-write-tree: error reading cache");
 
-	/* Verify that the tree is merged */
-	funny = 0;
-	for (i = 0; i < entries; i++) {
-		struct cache_entry *ce = active_cache[i];
-		if (ce_stage(ce)) {
-			if (10 < ++funny) {
-				fprintf(stderr, "...\n");
-				break;
-			}
-			fprintf(stderr, "%s: unmerged (%s)\n", ce->name, sha1_to_hex(ce->sha1));
+	if (!active_cache_tree)
+		active_cache_tree = cache_tree();
+
+	was_valid = cache_tree_fully_valid(active_cache_tree);
+	if (!was_valid) {
+		if (cache_tree_update(active_cache_tree,
+				      active_cache, active_nr,
+				      missing_ok, 0) < 0)
+			die("git-write-tree: error building trees");
+		if (0 <= newfd) {
+			if (!write_cache(newfd, active_cache, active_nr))
+				commit_index_file(&cache_file);
 		}
-	}
-	if (funny)
-		die("git-write-tree: not able to write tree");
-
-	/* Also verify that the cache does not have path and path/file
-	 * at the same time.  At this point we know the cache has only
-	 * stage 0 entries.
-	 */
-	funny = 0;
-	for (i = 0; i < entries - 1; i++) {
-		/* path/file always comes after path because of the way
-		 * the cache is sorted.  Also path can appear only once,
-		 * which means conflicting one would immediately follow.
+		/* Not being able to write is fine -- we are only interested
+		 * in updating the cache-tree part, and if the next caller
+		 * ends up using the old index with unupdated cache-tree part
+		 * it misses the work we did here, but that is just a
+		 * performance penalty and not a big deal.
 		 */
-		const char *this_name = active_cache[i]->name;
-		const char *next_name = active_cache[i+1]->name;
-		int this_len = strlen(this_name);
-		if (this_len < strlen(next_name) &&
-		    strncmp(this_name, next_name, this_len) == 0 &&
-		    next_name[this_len] == '/') {
-			if (10 < ++funny) {
-				fprintf(stderr, "...\n");
-				break;
-			}
-			fprintf(stderr, "You have both %s and %s\n",
-				this_name, next_name);
-		}
 	}
-	if (funny)
-		die("git-write-tree: not able to write tree");
-
-	/* Ok, write it out */
-	if (write_tree(active_cache, entries, "", 0, sha1) != entries)
-		die("git-write-tree: internal error");
-	printf("%s\n", sha1_to_hex(sha1));
+	if (prefix) {
+		struct cache_tree *subtree =
+			cache_tree_find(active_cache_tree, prefix);
+		printf("%s\n", sha1_to_hex(subtree->sha1));
+	}
+	else
+		printf("%s\n", sha1_to_hex(active_cache_tree->sha1));
 	return 0;
 }