From: Junio C Hamano <gitster@pobox.com>
Date: Mon, 26 Sep 2016 23:09:16 +0000 (-0700)
Subject: Merge branch 'mh/diff-indent-heuristic'
X-Git-Tag: v2.11.0-rc0~115
X-Git-Url: https://git.lorimer.id.au/gitweb.git/diff_plain/b7af6ae5cff8439fdf5b72f926cab2e614906af3?hp=-c

Merge branch 'mh/diff-indent-heuristic'

Output from "git diff" can be made easier to read by selecting
which lines are common and which lines are added/deleted
intelligently when the lines before and after the changed section
are the same.  A command line option is added to help with the
experiment to find a good heuristics.

* mh/diff-indent-heuristic:
blame: honor the diff heuristic options and config
parse-options: add parse_opt_unknown_cb()
diff: improve positioning of add/delete blocks in diffs
xdl_change_compact(): introduce the concept of a change group
recs_match(): take two xrecord_t pointers as arguments
is_blank_line(): take a single xrecord_t as argument
xdl_change_compact(): only use heuristic if group can't be matched
xdl_change_compact(): fix compaction heuristic to adjust ixo
---

b7af6ae5cff8439fdf5b72f926cab2e614906af3
diff --combined Documentation/diff-config.txt
index 0eded24034,ad368acda6..b27a38f896
--- a/Documentation/diff-config.txt
+++ b/Documentation/diff-config.txt
@@@ -105,7 -105,7 +105,7 @@@ diff.orderFile:
  
  diff.renameLimit::
  	The number of files to consider when performing the copy/rename
 -	detection; equivalent to the 'git diff' option '-l'.
 +	detection; equivalent to the 'git diff' option `-l`.
  
  diff.renames::
  	Whether and how Git detects renames.  If set to "false",
@@@ -122,11 -122,10 +122,11 @@@ diff.suppressBlankEmpty:
  
  diff.submodule::
  	Specify the format in which differences in submodules are
 -	shown.  The "log" format lists the commits in the range like
 -	linkgit:git-submodule[1] `summary` does.  The "short" format
 -	format just shows the names of the commits at the beginning
 -	and end of the range.  Defaults to short.
 +	shown.  The "short" format just shows the names of the commits
 +	at the beginning and end of the range. The "log" format lists
 +	the commits in the range like linkgit:git-submodule[1] `summary`
 +	does. The "diff" format shows an inline diff of the changed
 +	contents of the submodule. Defaults to "short".
  
  diff.wordRegex::
  	A POSIX Extended Regular Expression used to determine what is a "word"
@@@ -171,10 -170,11 +171,11 @@@ diff.tool:
  
  include::mergetools-diff.txt[]
  
+ diff.indentHeuristic::
  diff.compactionHeuristic::
- 	Set this option to `true` to enable an experimental heuristic that
- 	shifts the hunk boundary in an attempt to make the resulting
- 	patch easier to read.
+ 	Set one of these options to `true` to enable one of two
+ 	experimental heuristics that shift diff hunk boundaries to
+ 	make patches easier to read.
  
  diff.algorithm::
  	Choose a diff algorithm.  The variants are as follows:
diff --combined Documentation/diff-options.txt
index 7805a0ccad,0364061a4a..2d77a19626
--- a/Documentation/diff-options.txt
+++ b/Documentation/diff-options.txt
@@@ -63,12 -63,7 +63,7 @@@ ifndef::git-format-patch[
  	Synonym for `-p --raw`.
  endif::git-format-patch[]
  
- --compaction-heuristic::
- --no-compaction-heuristic::
- 	These are to help debugging and tuning an experimental
- 	heuristic (which is off by default) that shifts the hunk
- 	boundary in an attempt to make the resulting patch easier
- 	to read.
+ include::diff-heuristic-options.txt[]
  
  --minimal::
  	Spend extra time to make sure the smallest possible
@@@ -210,16 -205,13 +205,16 @@@ any of those replacements occurred
  	of the `--diff-filter` option on what the status letters mean.
  
  --submodule[=<format>]::
 -	Specify how differences in submodules are shown.  When `--submodule`
 -	or `--submodule=log` is given, the 'log' format is used.  This format lists
 -	the commits in the range like linkgit:git-submodule[1] `summary` does.
 -	Omitting the `--submodule` option or specifying `--submodule=short`,
 -	uses the 'short' format. This format just shows the names of the commits
 -	at the beginning and end of the range.  Can be tweaked via the
 -	`diff.submodule` configuration variable.
 +	Specify how differences in submodules are shown.  When specifying
 +	`--submodule=short` the 'short' format is used.  This format just
 +	shows the names of the commits at the beginning and end of the range.
 +	When `--submodule` or `--submodule=log` is specified, the 'log'
 +	format is used.  This format lists the commits in the range like
 +	linkgit:git-submodule[1] `summary` does.  When `--submodule=diff`
 +	is specified, the 'diff' format is used.  This format shows an
 +	inline diff of the changes in the submodule contents between the
 +	commit range.  Defaults to `diff.submodule` or the 'short' format
 +	if the config option is unset.
  
  --color[=<when>]::
  	Show colored diff.
@@@ -422,9 -414,6 +417,9 @@@ ifndef::git-format-patch[
  	paths are selected if there is any file that matches
  	other criteria in the comparison; if there is no file
  	that matches other criteria, nothing is selected.
 ++
 +Also, these upper-case letters can be downcased to exclude.  E.g.
 +`--diff-filter=ad` excludes added and deleted paths.
  
  -S<string>::
  	Look for differences that change the number of occurrences of
@@@ -572,8 -561,5 +567,8 @@@ endif::git-format-patch[
  --no-prefix::
  	Do not show any source or destination prefix.
  
 +--line-prefix=<prefix>::
 +	Prepend an additional prefix to every line of output.
 +
  For more detailed explanation on these common options, see also
  linkgit:gitdiffcore[7].
diff --combined builtin/blame.c
index 2ff18b168e,f82b0a0d94..a7bd7a6fd8
--- a/builtin/blame.c
+++ b/builtin/blame.c
@@@ -120,7 -120,7 +120,7 @@@ struct origin 
  	 */
  	struct blame_entry *suspects;
  	mmfile_t file;
 -	unsigned char blob_sha1[20];
 +	struct object_id blob_oid;
  	unsigned mode;
  	/* guilty gets set when shipping any suspects to the final
  	 * blame list instead of other commits
@@@ -134,7 -134,7 +134,7 @@@ struct progress_info 
  	int blamed_lines;
  };
  
 -static int diff_hunks(mmfile_t *file_a, mmfile_t *file_b, long ctxlen,
 +static int diff_hunks(mmfile_t *file_a, mmfile_t *file_b,
  		      xdl_emit_hunk_consume_func_t hunk_func, void *cb_data)
  {
  	xpparam_t xpp = {0};
@@@ -142,6 -142,7 +142,6 @@@
  	xdemitcb_t ecb = {NULL};
  
  	xpp.flags = xdl_opts;
 -	xecfg.ctxlen = ctxlen;
  	xecfg.hunk_func = hunk_func;
  	ecb.priv = cb_data;
  	return xdi_diff(file_a, file_b, &xpp, &xecfg, &ecb);
@@@ -154,8 -155,8 +154,8 @@@
   */
  int textconv_object(const char *path,
  		    unsigned mode,
 -		    const unsigned char *sha1,
 -		    int sha1_valid,
 +		    const struct object_id *oid,
 +		    int oid_valid,
  		    char **buf,
  		    unsigned long *buf_size)
  {
@@@ -163,7 -164,7 +163,7 @@@
  	struct userdiff_driver *textconv;
  
  	df = alloc_filespec(path);
 -	fill_filespec(df, sha1, sha1_valid, mode);
 +	fill_filespec(df, oid->hash, oid_valid, mode);
  	textconv = get_textconv(df);
  	if (!textconv) {
  		free_filespec(df);
@@@ -188,16 -189,15 +188,16 @@@ static void fill_origin_blob(struct dif
  
  		num_read_blob++;
  		if (DIFF_OPT_TST(opt, ALLOW_TEXTCONV) &&
 -		    textconv_object(o->path, o->mode, o->blob_sha1, 1, &file->ptr, &file_size))
 +		    textconv_object(o->path, o->mode, &o->blob_oid, 1, &file->ptr, &file_size))
  			;
  		else
 -			file->ptr = read_sha1_file(o->blob_sha1, &type, &file_size);
 +			file->ptr = read_sha1_file(o->blob_oid.hash, &type,
 +						   &file_size);
  		file->size = file_size;
  
  		if (!file->ptr)
  			die("Cannot read blob %s for path %s",
 -			    sha1_to_hex(o->blob_sha1),
 +			    oid_to_hex(&o->blob_oid),
  			    o->path);
  		o->file = *file;
  	}
@@@ -509,17 -509,17 +509,17 @@@ static struct origin *get_origin(struc
   */
  static int fill_blob_sha1_and_mode(struct origin *origin)
  {
 -	if (!is_null_sha1(origin->blob_sha1))
 +	if (!is_null_oid(&origin->blob_oid))
  		return 0;
  	if (get_tree_entry(origin->commit->object.oid.hash,
  			   origin->path,
 -			   origin->blob_sha1, &origin->mode))
 +			   origin->blob_oid.hash, &origin->mode))
  		goto error_out;
 -	if (sha1_object_info(origin->blob_sha1, NULL) != OBJ_BLOB)
 +	if (sha1_object_info(origin->blob_oid.hash, NULL) != OBJ_BLOB)
  		goto error_out;
  	return 0;
   error_out:
 -	hashclr(origin->blob_sha1);
 +	oidclr(&origin->blob_oid);
  	origin->mode = S_IFINVALID;
  	return -1;
  }
@@@ -573,7 -573,7 +573,7 @@@ static struct origin *find_origin(struc
  	if (!diff_queued_diff.nr) {
  		/* The path is the same as parent */
  		porigin = get_origin(sb, parent, origin->path);
 -		hashcpy(porigin->blob_sha1, origin->blob_sha1);
 +		oidcpy(&porigin->blob_oid, &origin->blob_oid);
  		porigin->mode = origin->mode;
  	} else {
  		/*
@@@ -599,7 -599,7 +599,7 @@@
  			    p->status);
  		case 'M':
  			porigin = get_origin(sb, parent, origin->path);
 -			hashcpy(porigin->blob_sha1, p->one->sha1);
 +			oidcpy(&porigin->blob_oid, &p->one->oid);
  			porigin->mode = p->one->mode;
  			break;
  		case 'A':
@@@ -645,7 -645,7 +645,7 @@@ static struct origin *find_rename(struc
  		if ((p->status == 'R' || p->status == 'C') &&
  		    !strcmp(p->two->path, origin->path)) {
  			porigin = get_origin(sb, parent, p->one->path);
 -			hashcpy(porigin->blob_sha1, p->one->sha1);
 +			oidcpy(&porigin->blob_oid, &p->one->oid);
  			porigin->mode = p->one->mode;
  			break;
  		}
@@@ -980,7 -980,7 +980,7 @@@ static void pass_blame_to_parent(struc
  	fill_origin_blob(&sb->revs->diffopt, target, &file_o);
  	num_get_patch++;
  
 -	if (diff_hunks(&file_p, &file_o, 0, blame_chunk_cb, &d))
 +	if (diff_hunks(&file_p, &file_o, blame_chunk_cb, &d))
  		die("unable to generate diff (%s -> %s)",
  		    oid_to_hex(&parent->commit->object.oid),
  		    oid_to_hex(&target->commit->object.oid));
@@@ -1129,7 -1129,7 +1129,7 @@@ static void find_copy_in_blob(struct sc
  	 * file_p partially may match that image.
  	 */
  	memset(split, 0, sizeof(struct blame_entry [3]));
 -	if (diff_hunks(file_p, &file_o, 1, handle_split_cb, &d))
 +	if (diff_hunks(file_p, &file_o, handle_split_cb, &d))
  		die("unable to generate diff (%s)",
  		    oid_to_hex(&parent->commit->object.oid));
  	/* remainder, if any, all match the preimage */
@@@ -1309,7 -1309,7 +1309,7 @@@ static void find_copy_in_parent(struct 
  				continue;
  
  			norigin = get_origin(sb, parent, p->one->path);
 -			hashcpy(norigin->blob_sha1, p->one->sha1);
 +			oidcpy(&norigin->blob_oid, &p->one->oid);
  			norigin->mode = p->one->mode;
  			fill_origin_blob(&sb->revs->diffopt, norigin, &file_p);
  			if (!file_p.ptr)
@@@ -1459,14 -1459,15 +1459,14 @@@ static void pass_blame(struct scoreboar
  			porigin = find(sb, p, origin);
  			if (!porigin)
  				continue;
 -			if (!hashcmp(porigin->blob_sha1, origin->blob_sha1)) {
 +			if (!oidcmp(&porigin->blob_oid, &origin->blob_oid)) {
  				pass_whole_blame(sb, origin, porigin);
  				origin_decref(porigin);
  				goto finish;
  			}
  			for (j = same = 0; j < i; j++)
  				if (sg_origin[j] &&
 -				    !hashcmp(sg_origin[j]->blob_sha1,
 -					     porigin->blob_sha1)) {
 +				    !oidcmp(&sg_origin[j]->blob_oid, &porigin->blob_oid)) {
  					same = 1;
  					break;
  				}
@@@ -1941,7 -1942,7 +1941,7 @@@ static void emit_other(struct scoreboar
  	cp = nth_line(sb, ent->lno);
  	for (cnt = 0; cnt < ent->num_lines; cnt++) {
  		char ch;
 -		int length = (opt & OUTPUT_LONG_OBJECT_NAME) ? 40 : abbrev;
 +		int length = (opt & OUTPUT_LONG_OBJECT_NAME) ? GIT_SHA1_HEXSZ : abbrev;
  
  		if (suspect->commit->object.flags & UNINTERESTING) {
  			if (blank_boundary)
@@@ -2220,6 -2221,8 +2220,8 @@@ static int git_blame_config(const char 
  		return 0;
  	}
  
+ 	if (git_diff_heuristic_config(var, value, cb) < 0)
+ 		return -1;
  	if (userdiff_config(var, value) < 0)
  		return -1;
  
@@@ -2229,35 -2232,26 +2231,35 @@@
  static void verify_working_tree_path(struct commit *work_tree, const char *path)
  {
  	struct commit_list *parents;
 +	int pos;
  
  	for (parents = work_tree->parents; parents; parents = parents->next) {
 -		const unsigned char *commit_sha1 = parents->item->object.oid.hash;
 -		unsigned char blob_sha1[20];
 +		const struct object_id *commit_oid = &parents->item->object.oid;
 +		struct object_id blob_oid;
  		unsigned mode;
  
 -		if (!get_tree_entry(commit_sha1, path, blob_sha1, &mode) &&
 -		    sha1_object_info(blob_sha1, NULL) == OBJ_BLOB)
 +		if (!get_tree_entry(commit_oid->hash, path, blob_oid.hash, &mode) &&
 +		    sha1_object_info(blob_oid.hash, NULL) == OBJ_BLOB)
  			return;
  	}
 -	die("no such path '%s' in HEAD", path);
 +
 +	pos = cache_name_pos(path, strlen(path));
 +	if (pos >= 0)
 +		; /* path is in the index */
 +	else if (-1 - pos < active_nr &&
 +		 !strcmp(active_cache[-1 - pos]->name, path))
 +		; /* path is in the index, unmerged */
 +	else
 +		die("no such path '%s' in HEAD", path);
  }
  
 -static struct commit_list **append_parent(struct commit_list **tail, const unsigned char *sha1)
 +static struct commit_list **append_parent(struct commit_list **tail, const struct object_id *oid)
  {
  	struct commit *parent;
  
 -	parent = lookup_commit_reference(sha1);
 +	parent = lookup_commit_reference(oid->hash);
  	if (!parent)
 -		die("no such commit %s", sha1_to_hex(sha1));
 +		die("no such commit %s", oid_to_hex(oid));
  	return &commit_list_insert(parent, tail)->next;
  }
  
@@@ -2274,10 -2268,10 +2276,10 @@@ static void append_merge_parents(struc
  	}
  
  	while (!strbuf_getwholeline_fd(&line, merge_head, '\n')) {
 -		unsigned char sha1[20];
 -		if (line.len < 40 || get_sha1_hex(line.buf, sha1))
 +		struct object_id oid;
 +		if (line.len < GIT_SHA1_HEXSZ || get_oid_hex(line.buf, &oid))
  			die("unknown line in '%s': %s", git_path_merge_head(), line.buf);
 -		tail = append_parent(tail, sha1);
 +		tail = append_parent(tail, &oid);
  	}
  	close(merge_head);
  	strbuf_release(&line);
@@@ -2306,7 -2300,7 +2308,7 @@@ static struct commit *fake_working_tree
  	struct commit *commit;
  	struct origin *origin;
  	struct commit_list **parent_tail, *parent;
 -	unsigned char head_sha1[20];
 +	struct object_id head_oid;
  	struct strbuf buf = STRBUF_INIT;
  	const char *ident;
  	time_t now;
@@@ -2322,10 -2316,10 +2324,10 @@@
  	commit->date = now;
  	parent_tail = &commit->parents;
  
 -	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, head_sha1, NULL))
 +	if (!resolve_ref_unsafe("HEAD", RESOLVE_REF_READING, head_oid.hash, NULL))
  		die("no such ref: HEAD");
  
 -	parent_tail = append_parent(parent_tail, head_sha1);
 +	parent_tail = append_parent(parent_tail, &head_oid);
  	append_merge_parents(parent_tail);
  	verify_working_tree_path(commit, path);
  
@@@ -2366,7 -2360,7 +2368,7 @@@
  		switch (st.st_mode & S_IFMT) {
  		case S_IFREG:
  			if (DIFF_OPT_TST(opt, ALLOW_TEXTCONV) &&
 -			    textconv_object(read_from, mode, null_sha1, 0, &buf_ptr, &buf_len))
 +			    textconv_object(read_from, mode, &null_oid, 0, &buf_ptr, &buf_len))
  				strbuf_attach(&buf, buf_ptr, buf_len, buf_len + 1);
  			else if (strbuf_read_file(&buf, read_from, st.st_size) != st.st_size)
  				die_errno("cannot open or read '%s'", read_from);
@@@ -2388,7 -2382,7 +2390,7 @@@
  	convert_to_git(path, buf.buf, buf.len, &buf, 0);
  	origin->file.ptr = buf.buf;
  	origin->file.size = buf.len;
 -	pretend_sha1_file(buf.buf, buf.len, OBJ_BLOB, origin->blob_sha1);
 +	pretend_sha1_file(buf.buf, buf.len, OBJ_BLOB, origin->blob_oid.hash);
  
  	/*
  	 * Read the current index, replace the path entry with
@@@ -2410,7 -2404,7 +2412,7 @@@
  	}
  	size = cache_entry_size(len);
  	ce = xcalloc(1, size);
 -	hashcpy(ce->sha1, origin->blob_sha1);
 +	oidcpy(&ce->oid, &origin->blob_oid);
  	memcpy(ce->name, path, len);
  	ce->ce_flags = create_ce_flags(0);
  	ce->ce_namelen = len;
@@@ -2550,6 -2544,15 +2552,15 @@@ int cmd_blame(int argc, const char **ar
  		OPT_BIT('s', NULL, &output_option, N_("Suppress author name and timestamp (Default: off)"), OUTPUT_NO_AUTHOR),
  		OPT_BIT('e', "show-email", &output_option, N_("Show author email instead of name (Default: off)"), OUTPUT_SHOW_EMAIL),
  		OPT_BIT('w', NULL, &xdl_opts, N_("Ignore whitespace differences"), XDF_IGNORE_WHITESPACE),
+ 
+ 		/*
+ 		 * The following two options are parsed by parse_revision_opt()
+ 		 * and are only included here to get included in the "-h"
+ 		 * output:
+ 		 */
+ 		{ OPTION_LOWLEVEL_CALLBACK, 0, "indent-heuristic", NULL, NULL, N_("Use an experimental indent-based heuristic to improve diffs"), PARSE_OPT_NOARG, parse_opt_unknown_cb },
+ 		{ OPTION_LOWLEVEL_CALLBACK, 0, "compaction-heuristic", NULL, NULL, N_("Use an experimental blank-line-based heuristic to improve diffs"), PARSE_OPT_NOARG, parse_opt_unknown_cb },
+ 
  		OPT_BIT(0, "minimal", &xdl_opts, N_("Spend extra cycles to find better match"), XDF_NEED_MINIMAL),
  		OPT_STRING('S', NULL, &revs_file, N_("file"), N_("Use revisions from <file> instead of calling git-rev-list")),
  		OPT_STRING(0, "contents", &contents_from, N_("file"), N_("Use <file>'s contents as the final image")),
@@@ -2596,12 -2599,13 +2607,13 @@@
  	}
  parse_done:
  	no_whole_file_rename = !DIFF_OPT_TST(&revs.diffopt, FOLLOW_RENAMES);
+ 	xdl_opts |= revs.diffopt.xdl_opts & (XDF_COMPACTION_HEURISTIC | XDF_INDENT_HEURISTIC);
  	DIFF_OPT_CLR(&revs.diffopt, FOLLOW_RENAMES);
  	argc = parse_options_end(&ctx);
  
  	if (incremental || (output_option & OUTPUT_PORCELAIN)) {
  		if (show_progress > 0)
 -			die("--progress can't be used with --incremental or porcelain formats");
 +			die(_("--progress can't be used with --incremental or porcelain formats"));
  		show_progress = 0;
  	} else if (show_progress < 0)
  		show_progress = isatty(2);
@@@ -2634,9 -2638,6 +2646,9 @@@
  	case DATE_RAW:
  		blame_date_width = sizeof("1161298804 -0700");
  		break;
 +	case DATE_UNIX:
 +		blame_date_width = sizeof("1161298804");
 +		break;
  	case DATE_SHORT:
  		blame_date_width = sizeof("2006-10-19");
  		break;
@@@ -2727,7 -2728,7 +2739,7 @@@
  		sb.commits.compare = compare_commits_by_commit_date;
  	}
  	else if (contents_from)
 -		die("--contents and --reverse do not blend well.");
 +		die(_("--contents and --reverse do not blend well."));
  	else {
  		final_commit_name = prepare_initial(&sb);
  		sb.commits.compare = compare_commits_by_reverse_commit_date;
@@@ -2747,12 -2748,12 +2759,12 @@@
  		add_pending_object(&revs, &(sb.final->object), ":");
  	}
  	else if (contents_from)
 -		die("Cannot use --contents with final commit object name");
 +		die(_("cannot use --contents with final commit object name"));
  
  	if (reverse && revs.first_parent_only) {
  		final_commit = find_single_final(sb.revs, NULL);
  		if (!final_commit)
 -			die("--reverse and --first-parent together require specified latest commit");
 +			die(_("--reverse and --first-parent together require specified latest commit"));
  	}
  
  	/*
@@@ -2779,7 -2780,7 +2791,7 @@@
  		}
  
  		if (oidcmp(&c->object.oid, &sb.final->object.oid))
 -			die("--reverse --first-parent together require range along first-parent chain");
 +			die(_("--reverse --first-parent together require range along first-parent chain"));
  	}
  
  	if (is_null_oid(&sb.final->object.oid)) {
@@@ -2790,26 -2791,26 +2802,26 @@@
  	else {
  		o = get_origin(&sb, sb.final, path);
  		if (fill_blob_sha1_and_mode(o))
 -			die("no such path %s in %s", path, final_commit_name);
 +			die(_("no such path %s in %s"), path, final_commit_name);
  
  		if (DIFF_OPT_TST(&sb.revs->diffopt, ALLOW_TEXTCONV) &&
 -		    textconv_object(path, o->mode, o->blob_sha1, 1, (char **) &sb.final_buf,
 +		    textconv_object(path, o->mode, &o->blob_oid, 1, (char **) &sb.final_buf,
  				    &sb.final_buf_size))
  			;
  		else
 -			sb.final_buf = read_sha1_file(o->blob_sha1, &type,
 +			sb.final_buf = read_sha1_file(o->blob_oid.hash, &type,
  						      &sb.final_buf_size);
  
  		if (!sb.final_buf)
 -			die("Cannot read blob %s for path %s",
 -			    sha1_to_hex(o->blob_sha1),
 +			die(_("cannot read blob %s for path %s"),
 +			    oid_to_hex(&o->blob_oid),
  			    path);
  	}
  	num_read_blob++;
  	lno = prepare_lines(&sb);
  
  	if (lno && !range_list.nr)
 -		string_list_append(&range_list, xstrdup("1"));
 +		string_list_append(&range_list, "1");
  
  	anchor = 1;
  	range_set_init(&ranges, range_list.nr);
@@@ -2820,9 -2821,7 +2832,9 @@@
  				    &bottom, &top, sb.path))
  			usage(blame_usage);
  		if (lno < top || ((lno || bottom) && lno < bottom))
 -			die("file %s has only %lu lines", path, lno);
 +			die(Q_("file %s has only %lu line",
 +			       "file %s has only %lu lines",
 +			       lno), path, lno);
  		if (bottom < 1)
  			bottom = 1;
  		if (top < 1)
diff --combined diff.c
index c6da383c56,56a6dcac34..dc5ab8b4fb
--- a/diff.c
+++ b/diff.c
@@@ -18,7 -18,6 +18,7 @@@
  #include "ll-merge.h"
  #include "string-list.h"
  #include "argv-array.h"
 +#include "graph.h"
  
  #ifdef NO_FAST_WORKING_DIRECTORY
  #define FAST_WORKING_DIRECTORY 0
@@@ -27,6 -26,7 +27,7 @@@
  #endif
  
  static int diff_detect_rename_default;
+ static int diff_indent_heuristic; /* experimental */
  static int diff_compaction_heuristic; /* experimental */
  static int diff_rename_limit_default = 400;
  static int diff_suppress_blank_empty;
@@@ -132,11 -132,9 +133,11 @@@ static int parse_dirstat_params(struct 
  static int parse_submodule_params(struct diff_options *options, const char *value)
  {
  	if (!strcmp(value, "log"))
 -		DIFF_OPT_SET(options, SUBMODULE_LOG);
 +		options->submodule_format = DIFF_SUBMODULE_LOG;
  	else if (!strcmp(value, "short"))
 -		DIFF_OPT_CLR(options, SUBMODULE_LOG);
 +		options->submodule_format = DIFF_SUBMODULE_SHORT;
 +	else if (!strcmp(value, "diff"))
 +		options->submodule_format = DIFF_SUBMODULE_INLINE_DIFF;
  	else
  		return -1;
  	return 0;
@@@ -177,6 -175,21 +178,21 @@@ void init_diff_ui_defaults(void
  	diff_detect_rename_default = 1;
  }
  
+ int git_diff_heuristic_config(const char *var, const char *value, void *cb)
+ {
+ 	if (!strcmp(var, "diff.indentheuristic")) {
+ 		diff_indent_heuristic = git_config_bool(var, value);
+ 		if (diff_indent_heuristic)
+ 			diff_compaction_heuristic = 0;
+ 	}
+ 	if (!strcmp(var, "diff.compactionheuristic")) {
+ 		diff_compaction_heuristic = git_config_bool(var, value);
+ 		if (diff_compaction_heuristic)
+ 			diff_indent_heuristic = 0;
+ 	}
+ 	return 0;
+ }
+ 
  int git_diff_ui_config(const char *var, const char *value, void *cb)
  {
  	if (!strcmp(var, "diff.color") || !strcmp(var, "color.diff")) {
@@@ -193,10 -206,6 +209,6 @@@
  		diff_detect_rename_default = git_config_rename(var, value);
  		return 0;
  	}
- 	if (!strcmp(var, "diff.compactionheuristic")) {
- 		diff_compaction_heuristic = git_config_bool(var, value);
- 		return 0;
- 	}
  	if (!strcmp(var, "diff.autorefreshindex")) {
  		diff_auto_refresh_index = git_config_bool(var, value);
  		return 0;
@@@ -237,6 -246,8 +249,8 @@@
  		return 0;
  	}
  
+ 	if (git_diff_heuristic_config(var, value, cb) < 0)
+ 		return -1;
  	if (git_color_config(var, value, cb) < 0)
  		return -1;
  
@@@ -357,6 -368,7 +371,6 @@@ struct emit_callback 
  	const char **label_path;
  	struct diff_words_data *diff_words;
  	struct diff_options *opt;
 -	int *found_changesp;
  	struct strbuf *header;
  };
  
@@@ -724,6 -736,7 +738,6 @@@ static void emit_rewrite_diff(const cha
  
  	memset(&ecbdata, 0, sizeof(ecbdata));
  	ecbdata.color_diff = want_color(o->use_color);
 -	ecbdata.found_changesp = &o->found_changes;
  	ecbdata.ws_rule = whitespace_rule(name_b);
  	ecbdata.opt = o;
  	if (ecbdata.ws_rule & WS_BLANK_AT_EOF) {
@@@ -1217,13 -1230,12 +1231,13 @@@ static void fn_out_consume(void *priv, 
  	struct diff_options *o = ecbdata->opt;
  	const char *line_prefix = diff_line_prefix(o);
  
 +	o->found_changes = 1;
 +
  	if (ecbdata->header) {
 -		fprintf(ecbdata->opt->file, "%s", ecbdata->header->buf);
 +		fprintf(o->file, "%s", ecbdata->header->buf);
  		strbuf_reset(ecbdata->header);
  		ecbdata->header = NULL;
  	}
 -	*(ecbdata->found_changesp) = 1;
  
  	if (ecbdata->label_path[0]) {
  		const char *name_a_tab, *name_b_tab;
@@@ -1231,9 -1243,9 +1245,9 @@@
  		name_a_tab = strchr(ecbdata->label_path[0], ' ') ? "\t" : "";
  		name_b_tab = strchr(ecbdata->label_path[1], ' ') ? "\t" : "";
  
 -		fprintf(ecbdata->opt->file, "%s%s--- %s%s%s\n",
 +		fprintf(o->file, "%s%s--- %s%s%s\n",
  			line_prefix, meta, ecbdata->label_path[0], reset, name_a_tab);
 -		fprintf(ecbdata->opt->file, "%s%s+++ %s%s%s\n",
 +		fprintf(o->file, "%s%s+++ %s%s%s\n",
  			line_prefix, meta, ecbdata->label_path[1], reset, name_b_tab);
  		ecbdata->label_path[0] = ecbdata->label_path[1] = NULL;
  	}
@@@ -1251,7 -1263,15 +1265,7 @@@
  		find_lno(line, ecbdata);
  		emit_hunk_header(ecbdata, line, len);
  		if (line[len-1] != '\n')
 -			putc('\n', ecbdata->opt->file);
 -		return;
 -	}
 -
 -	if (len < 1) {
 -		emit_line(ecbdata->opt, reset, reset, line, len);
 -		if (ecbdata->diff_words
 -		    && ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN)
 -			fputs("~\n", ecbdata->opt->file);
 +			putc('\n', o->file);
  		return;
  	}
  
@@@ -1276,8 -1296,8 +1290,8 @@@
  		}
  		diff_words_flush(ecbdata);
  		if (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {
 -			emit_line(ecbdata->opt, context, reset, line, len);
 -			fputs("~\n", ecbdata->opt->file);
 +			emit_line(o, context, reset, line, len);
 +			fputs("~\n", o->file);
  		} else {
  			/*
  			 * Skip the prefix character, if any.  With
@@@ -1288,7 -1308,7 +1302,7 @@@
  			      line++;
  			      len--;
  			}
 -			emit_line(ecbdata->opt, context, reset, line, len);
 +			emit_line(o, context, reset, line, len);
  		}
  		return;
  	}
@@@ -1310,7 -1330,8 +1324,7 @@@
  	default:
  		/* incomplete line at the end */
  		ecbdata->lno_in_preimage++;
 -		emit_line(ecbdata->opt,
 -			  diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),
 +		emit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),
  			  reset, line, len);
  		break;
  	}
@@@ -1618,7 -1639,7 +1632,7 @@@ static void show_stats(struct diffstat_
  	 */
  
  	if (options->stat_width == -1)
 -		width = term_columns() - options->output_prefix_length;
 +		width = term_columns() - strlen(line_prefix);
  	else
  		width = options->stat_width ? options->stat_width : 80;
  	number_width = decimal_width(max_change) > number_width ?
@@@ -1926,8 -1947,8 +1940,8 @@@ static void show_dirstat(struct diff_op
  
  		name = p->two->path ? p->two->path : p->one->path;
  
 -		if (p->one->sha1_valid && p->two->sha1_valid)
 -			content_changed = hashcmp(p->one->sha1, p->two->sha1);
 +		if (p->one->oid_valid && p->two->oid_valid)
 +			content_changed = oidcmp(&p->one->oid, &p->two->oid);
  		else
  			content_changed = 1;
  
@@@ -2292,37 -2313,16 +2306,37 @@@ static void builtin_diff(const char *na
  	struct strbuf header = STRBUF_INIT;
  	const char *line_prefix = diff_line_prefix(o);
  
 -	if (DIFF_OPT_TST(o, SUBMODULE_LOG) &&
 -			(!one->mode || S_ISGITLINK(one->mode)) &&
 -			(!two->mode || S_ISGITLINK(two->mode))) {
 +	diff_set_mnemonic_prefix(o, "a/", "b/");
 +	if (DIFF_OPT_TST(o, REVERSE_DIFF)) {
 +		a_prefix = o->b_prefix;
 +		b_prefix = o->a_prefix;
 +	} else {
 +		a_prefix = o->a_prefix;
 +		b_prefix = o->b_prefix;
 +	}
 +
 +	if (o->submodule_format == DIFF_SUBMODULE_LOG &&
 +	    (!one->mode || S_ISGITLINK(one->mode)) &&
 +	    (!two->mode || S_ISGITLINK(two->mode))) {
  		const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
  		const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
  		show_submodule_summary(o->file, one->path ? one->path : two->path,
  				line_prefix,
 -				one->sha1, two->sha1, two->dirty_submodule,
 +				&one->oid, &two->oid,
 +				two->dirty_submodule,
  				meta, del, add, reset);
  		return;
 +	} else if (o->submodule_format == DIFF_SUBMODULE_INLINE_DIFF &&
 +		   (!one->mode || S_ISGITLINK(one->mode)) &&
 +		   (!two->mode || S_ISGITLINK(two->mode))) {
 +		const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
 +		const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
 +		show_submodule_inline_diff(o->file, one->path ? one->path : two->path,
 +				line_prefix,
 +				&one->oid, &two->oid,
 +				two->dirty_submodule,
 +				meta, del, add, reset, o);
 +		return;
  	}
  
  	if (DIFF_OPT_TST(o, ALLOW_TEXTCONV)) {
@@@ -2330,6 -2330,15 +2344,6 @@@
  		textconv_two = get_textconv(two);
  	}
  
 -	diff_set_mnemonic_prefix(o, "a/", "b/");
 -	if (DIFF_OPT_TST(o, REVERSE_DIFF)) {
 -		a_prefix = o->b_prefix;
 -		b_prefix = o->a_prefix;
 -	} else {
 -		a_prefix = o->a_prefix;
 -		b_prefix = o->b_prefix;
 -	}
 -
  	/* Never use a non-valid filename anywhere if at all possible */
  	name_a = DIFF_FILE_VALID(one) ? name_a : name_b;
  	name_b = DIFF_FILE_VALID(two) ? name_b : name_a;
@@@ -2389,7 -2398,7 +2403,7 @@@
  		if (!one->data && !two->data &&
  		    S_ISREG(one->mode) && S_ISREG(two->mode) &&
  		    !DIFF_OPT_TST(o, BINARY)) {
 -			if (!hashcmp(one->sha1, two->sha1)) {
 +			if (!oidcmp(&one->oid, &two->oid)) {
  				if (must_show_header)
  					fprintf(o->file, "%s", header.buf);
  				goto free_ab_and_return;
@@@ -2442,6 -2451,7 +2456,6 @@@
  		memset(&ecbdata, 0, sizeof(ecbdata));
  		ecbdata.label_path = lbl;
  		ecbdata.color_diff = want_color(o->use_color);
 -		ecbdata.found_changesp = &o->found_changes;
  		ecbdata.ws_rule = whitespace_rule(name_b);
  		if (ecbdata.ws_rule & WS_BLANK_AT_EOF)
  			check_blank_at_eof(&mf1, &mf2, &ecbdata);
@@@ -2509,7 -2519,7 +2523,7 @@@ static void builtin_diffstat(const cha
  		return;
  	}
  
 -	same_contents = !hashcmp(one->sha1, two->sha1);
 +	same_contents = !oidcmp(&one->oid, &two->oid);
  
  	if (diff_filespec_is_binary(one) || diff_filespec_is_binary(two)) {
  		data->is_binary = 1;
@@@ -2642,8 -2652,8 +2656,8 @@@ void fill_filespec(struct diff_filespe
  {
  	if (mode) {
  		spec->mode = canon_mode(mode);
 -		hashcpy(spec->sha1, sha1);
 -		spec->sha1_valid = sha1_valid;
 +		hashcpy(spec->oid.hash, sha1);
 +		spec->oid_valid = sha1_valid;
  	}
  }
  
@@@ -2686,13 -2696,6 +2700,13 @@@ static int reuse_worktree_file(const ch
  	if (!FAST_WORKING_DIRECTORY && !want_file && has_sha1_pack(sha1))
  		return 0;
  
 +	/*
 +	 * Similarly, if we'd have to convert the file contents anyway, that
 +	 * makes the optimization not worthwhile.
 +	 */
 +	if (!want_file && would_convert_to_git(name))
 +		return 0;
 +
  	len = strlen(name);
  	pos = cache_name_pos(name, len);
  	if (pos < 0)
@@@ -2703,7 -2706,7 +2717,7 @@@
  	 * This is not the sha1 we are looking for, or
  	 * unreusable because it is not a regular file.
  	 */
 -	if (hashcmp(sha1, ce->sha1) || !S_ISREG(ce->ce_mode))
 +	if (hashcmp(sha1, ce->oid.hash) || !S_ISREG(ce->ce_mode))
  		return 0;
  
  	/*
@@@ -2732,8 -2735,7 +2746,8 @@@ static int diff_populate_gitlink(struc
  	if (s->dirty_submodule)
  		dirty = "-dirty";
  
 -	strbuf_addf(&buf, "Subproject commit %s%s\n", sha1_to_hex(s->sha1), dirty);
 +	strbuf_addf(&buf, "Subproject commit %s%s\n",
 +		    oid_to_hex(&s->oid), dirty);
  	s->size = buf.len;
  	if (size_only) {
  		s->data = NULL;
@@@ -2776,8 -2778,8 +2790,8 @@@ int diff_populate_filespec(struct diff_
  	if (S_ISGITLINK(s->mode))
  		return diff_populate_gitlink(s, size_only);
  
 -	if (!s->sha1_valid ||
 -	    reuse_worktree_file(s->path, s->sha1, 0)) {
 +	if (!s->oid_valid ||
 +	    reuse_worktree_file(s->path, s->oid.hash, 0)) {
  		struct strbuf buf = STRBUF_INIT;
  		struct stat st;
  		int fd;
@@@ -2834,10 -2836,9 +2848,10 @@@
  	else {
  		enum object_type type;
  		if (size_only || (flags & CHECK_BINARY)) {
 -			type = sha1_object_info(s->sha1, &s->size);
 +			type = sha1_object_info(s->oid.hash, &s->size);
  			if (type < 0)
 -				die("unable to read %s", sha1_to_hex(s->sha1));
 +				die("unable to read %s",
 +				    oid_to_hex(&s->oid));
  			if (size_only)
  				return 0;
  			if (s->size > big_file_threshold && s->is_binary == -1) {
@@@ -2845,9 -2846,9 +2859,9 @@@
  				return 0;
  			}
  		}
 -		s->data = read_sha1_file(s->sha1, &type, &s->size);
 +		s->data = read_sha1_file(s->oid.hash, &type, &s->size);
  		if (!s->data)
 -			die("unable to read %s", sha1_to_hex(s->sha1));
 +			die("unable to read %s", oid_to_hex(&s->oid));
  		s->should_free = 1;
  	}
  	return 0;
@@@ -2876,7 -2877,7 +2890,7 @@@ void diff_free_filespec_data(struct dif
  static void prep_temp_blob(const char *path, struct diff_tempfile *temp,
  			   void *blob,
  			   unsigned long size,
 -			   const unsigned char *sha1,
 +			   const struct object_id *oid,
  			   int mode)
  {
  	int fd;
@@@ -2901,7 -2902,7 +2915,7 @@@
  		die_errno("unable to write temp-file");
  	close_tempfile(&temp->tempfile);
  	temp->name = get_tempfile_path(&temp->tempfile);
 -	sha1_to_hex_r(temp->hex, sha1);
 +	oid_to_hex_r(temp->hex, oid);
  	xsnprintf(temp->mode, sizeof(temp->mode), "%06o", mode);
  	strbuf_release(&buf);
  	strbuf_release(&template);
@@@ -2925,8 -2926,8 +2939,8 @@@ static struct diff_tempfile *prepare_te
  	}
  
  	if (!S_ISGITLINK(one->mode) &&
 -	    (!one->sha1_valid ||
 -	     reuse_worktree_file(name, one->sha1, 1))) {
 +	    (!one->oid_valid ||
 +	     reuse_worktree_file(name, one->oid.hash, 1))) {
  		struct stat st;
  		if (lstat(name, &st) < 0) {
  			if (errno == ENOENT)
@@@ -2938,19 -2939,19 +2952,19 @@@
  			if (strbuf_readlink(&sb, name, st.st_size) < 0)
  				die_errno("readlink(%s)", name);
  			prep_temp_blob(name, temp, sb.buf, sb.len,
 -				       (one->sha1_valid ?
 -					one->sha1 : null_sha1),
 -				       (one->sha1_valid ?
 +				       (one->oid_valid ?
 +					&one->oid : &null_oid),
 +				       (one->oid_valid ?
  					one->mode : S_IFLNK));
  			strbuf_release(&sb);
  		}
  		else {
  			/* we can borrow from the file in the work tree */
  			temp->name = name;
 -			if (!one->sha1_valid)
 +			if (!one->oid_valid)
  				sha1_to_hex_r(temp->hex, null_sha1);
  			else
 -				sha1_to_hex_r(temp->hex, one->sha1);
 +				sha1_to_hex_r(temp->hex, one->oid.hash);
  			/* Even though we may sometimes borrow the
  			 * contents from the work tree, we always want
  			 * one->mode.  mode is trustworthy even when
@@@ -2965,7 -2966,7 +2979,7 @@@
  		if (diff_populate_filespec(one, 0))
  			die("cannot read data blob for %s", one->path);
  		prep_temp_blob(name, temp, one->data, one->size,
 -			       one->sha1, one->mode);
 +			       &one->oid, one->mode);
  	}
  	return temp;
  }
@@@ -3078,7 -3079,7 +3092,7 @@@ static void fill_metainfo(struct strbu
  	default:
  		*must_show_header = 0;
  	}
 -	if (one && two && hashcmp(one->sha1, two->sha1)) {
 +	if (one && two && oidcmp(&one->oid, &two->oid)) {
  		int abbrev = DIFF_OPT_TST(o, FULL_INDEX) ? 40 : DEFAULT_ABBREV;
  
  		if (DIFF_OPT_TST(o, BINARY)) {
@@@ -3088,8 -3089,8 +3102,8 @@@
  				abbrev = 40;
  		}
  		strbuf_addf(msg, "%s%sindex %s..", line_prefix, set,
 -			    find_unique_abbrev(one->sha1, abbrev));
 -		strbuf_addstr(msg, find_unique_abbrev(two->sha1, abbrev));
 +			    find_unique_abbrev(one->oid.hash, abbrev));
 +		strbuf_addstr(msg, find_unique_abbrev(two->oid.hash, abbrev));
  		if (one->mode == two->mode)
  			strbuf_addf(msg, " %06o", one->mode);
  		strbuf_addf(msg, "%s\n", reset);
@@@ -3144,20 -3145,20 +3158,20 @@@ static void run_diff_cmd(const char *pg
  static void diff_fill_sha1_info(struct diff_filespec *one)
  {
  	if (DIFF_FILE_VALID(one)) {
 -		if (!one->sha1_valid) {
 +		if (!one->oid_valid) {
  			struct stat st;
  			if (one->is_stdin) {
 -				hashcpy(one->sha1, null_sha1);
 +				oidclr(&one->oid);
  				return;
  			}
  			if (lstat(one->path, &st) < 0)
  				die_errno("stat '%s'", one->path);
 -			if (index_path(one->sha1, one->path, &st, 0))
 +			if (index_path(one->oid.hash, one->path, &st, 0))
  				die("cannot hash %s", one->path);
  		}
  	}
  	else
 -		hashclr(one->sha1);
 +		oidclr(&one->oid);
  }
  
  static void strip_prefix(int prefix_length, const char **namep, const char **otherp)
@@@ -3296,7 -3297,9 +3310,9 @@@ void diff_setup(struct diff_options *op
  	options->use_color = diff_use_color_default;
  	options->detect_rename = diff_detect_rename_default;
  	options->xdl_opts |= diff_algorithm;
- 	if (diff_compaction_heuristic)
+ 	if (diff_indent_heuristic)
+ 		DIFF_XDL_SET(options, INDENT_HEURISTIC);
+ 	else if (diff_compaction_heuristic)
  		DIFF_XDL_SET(options, COMPACTION_HEURISTIC);
  
  	options->orderfile = diff_order_file_cfg;
@@@ -3818,9 -3821,15 +3834,15 @@@ int diff_opt_parse(struct diff_options 
  		DIFF_XDL_SET(options, IGNORE_WHITESPACE_AT_EOL);
  	else if (!strcmp(arg, "--ignore-blank-lines"))
  		DIFF_XDL_SET(options, IGNORE_BLANK_LINES);
- 	else if (!strcmp(arg, "--compaction-heuristic"))
+ 	else if (!strcmp(arg, "--indent-heuristic")) {
+ 		DIFF_XDL_SET(options, INDENT_HEURISTIC);
+ 		DIFF_XDL_CLR(options, COMPACTION_HEURISTIC);
+ 	} else if (!strcmp(arg, "--no-indent-heuristic"))
+ 		DIFF_XDL_CLR(options, INDENT_HEURISTIC);
+ 	else if (!strcmp(arg, "--compaction-heuristic")) {
  		DIFF_XDL_SET(options, COMPACTION_HEURISTIC);
- 	else if (!strcmp(arg, "--no-compaction-heuristic"))
+ 		DIFF_XDL_CLR(options, INDENT_HEURISTIC);
+ 	} else if (!strcmp(arg, "--no-compaction-heuristic"))
  		DIFF_XDL_CLR(options, COMPACTION_HEURISTIC);
  	else if (!strcmp(arg, "--patience"))
  		options->xdl_opts = DIFF_WITH_ALG(options, PATIENCE_DIFF);
@@@ -3918,7 -3927,7 +3940,7 @@@
  		DIFF_OPT_SET(options, OVERRIDE_SUBMODULE_CONFIG);
  		handle_ignore_submodules_arg(options, arg);
  	} else if (!strcmp(arg, "--submodule"))
 -		DIFF_OPT_SET(options, SUBMODULE_LOG);
 +		options->submodule_format = DIFF_SUBMODULE_LOG;
  	else if (skip_prefix(arg, "--submodule=", &arg))
  		return parse_submodule_opt(options, arg);
  	else if (skip_prefix(arg, "--ws-error-highlight=", &arg))
@@@ -3969,12 -3978,6 +3991,12 @@@
  		options->a_prefix = optarg;
  		return argcount;
  	}
 +	else if ((argcount = parse_long_opt("line-prefix", av, &optarg))) {
 +		options->line_prefix = optarg;
 +		options->line_prefix_length = strlen(options->line_prefix);
 +		graph_setup_line_prefix(options);
 +		return argcount;
 +	}
  	else if ((argcount = parse_long_opt("dst-prefix", av, &optarg))) {
  		options->b_prefix = optarg;
  		return argcount;
@@@ -3996,8 -3999,6 +4018,8 @@@
  		if (!options->file)
  			die_errno("Could not open '%s'", path);
  		options->close_file = 1;
 +		if (options->use_color != GIT_COLOR_ALWAYS)
 +			options->use_color = GIT_COLOR_NEVER;
  		return argcount;
  	} else
  		return 0;
@@@ -4139,9 -4140,8 +4161,9 @@@ static void diff_flush_raw(struct diff_
  	fprintf(opt->file, "%s", diff_line_prefix(opt));
  	if (!(opt->output_format & DIFF_FORMAT_NAME_STATUS)) {
  		fprintf(opt->file, ":%06o %06o %s ", p->one->mode, p->two->mode,
 -			diff_unique_abbrev(p->one->sha1, opt->abbrev));
 -		fprintf(opt->file, "%s ", diff_unique_abbrev(p->two->sha1, opt->abbrev));
 +			diff_unique_abbrev(p->one->oid.hash, opt->abbrev));
 +		fprintf(opt->file, "%s ",
 +			diff_unique_abbrev(p->two->oid.hash, opt->abbrev));
  	}
  	if (p->score) {
  		fprintf(opt->file, "%c%03d%c", p->status, similarity_index(p),
@@@ -4190,11 -4190,11 +4212,11 @@@ int diff_unmodified_pair(struct diff_fi
  	/* both are valid and point at the same path.  that is, we are
  	 * dealing with a change.
  	 */
 -	if (one->sha1_valid && two->sha1_valid &&
 -	    !hashcmp(one->sha1, two->sha1) &&
 +	if (one->oid_valid && two->oid_valid &&
 +	    !oidcmp(&one->oid, &two->oid) &&
  	    !one->dirty_submodule && !two->dirty_submodule)
  		return 1; /* no change */
 -	if (!one->sha1_valid && !two->sha1_valid)
 +	if (!one->oid_valid && !two->oid_valid)
  		return 1; /* both look at the same file on the filesystem. */
  	return 0;
  }
@@@ -4255,7 -4255,7 +4277,7 @@@ void diff_debug_filespec(struct diff_fi
  		s->path,
  		DIFF_FILE_VALID(s) ? "valid" : "invalid",
  		s->mode,
 -		s->sha1_valid ? sha1_to_hex(s->sha1) : "");
 +		s->oid_valid ? oid_to_hex(&s->oid) : "");
  	fprintf(stderr, "queue[%d] %s size %lu\n",
  		x, one ? one : "",
  		s->size);
@@@ -4325,11 -4325,11 +4347,11 @@@ static void diff_resolve_rename_copy(vo
  			else
  				p->status = DIFF_STATUS_RENAMED;
  		}
 -		else if (hashcmp(p->one->sha1, p->two->sha1) ||
 +		else if (oidcmp(&p->one->oid, &p->two->oid) ||
  			 p->one->mode != p->two->mode ||
  			 p->one->dirty_submodule ||
  			 p->two->dirty_submodule ||
 -			 is_null_sha1(p->one->sha1))
 +			 is_null_oid(&p->one->oid))
  			p->status = DIFF_STATUS_MODIFIED;
  		else {
  			/* This is a "no-change" entry and should not
@@@ -4471,7 -4471,7 +4493,7 @@@ static void patch_id_consume(void *priv
  }
  
  /* returns 0 upon success, and writes result into sha1 */
 -static int diff_get_patch_id(struct diff_options *options, unsigned char *sha1)
 +static int diff_get_patch_id(struct diff_options *options, unsigned char *sha1, int diff_header_only)
  {
  	struct diff_queue_struct *q = &diff_queued_diff;
  	int i;
@@@ -4506,6 -4506,9 +4528,6 @@@
  
  		diff_fill_sha1_info(p->one);
  		diff_fill_sha1_info(p->two);
 -		if (fill_mmfile(&mf1, p->one) < 0 ||
 -				fill_mmfile(&mf2, p->two) < 0)
 -			return error("unable to read files to diff");
  
  		len1 = remove_space(p->one->path, strlen(p->one->path));
  		len2 = remove_space(p->two->path, strlen(p->two->path));
@@@ -4540,19 -4543,10 +4562,19 @@@
  					len2, p->two->path);
  		git_SHA1_Update(&ctx, buffer, len1);
  
 +		if (diff_header_only)
 +			continue;
 +
 +		if (fill_mmfile(&mf1, p->one) < 0 ||
 +		    fill_mmfile(&mf2, p->two) < 0)
 +			return error("unable to read files to diff");
 +
  		if (diff_filespec_is_binary(p->one) ||
  		    diff_filespec_is_binary(p->two)) {
 -			git_SHA1_Update(&ctx, sha1_to_hex(p->one->sha1), 40);
 -			git_SHA1_Update(&ctx, sha1_to_hex(p->two->sha1), 40);
 +			git_SHA1_Update(&ctx, oid_to_hex(&p->one->oid),
 +					40);
 +			git_SHA1_Update(&ctx, oid_to_hex(&p->two->oid),
 +					40);
  			continue;
  		}
  
@@@ -4569,11 -4563,11 +4591,11 @@@
  	return 0;
  }
  
 -int diff_flush_patch_id(struct diff_options *options, unsigned char *sha1)
 +int diff_flush_patch_id(struct diff_options *options, unsigned char *sha1, int diff_header_only)
  {
  	struct diff_queue_struct *q = &diff_queued_diff;
  	int i;
 -	int result = diff_get_patch_id(options, sha1);
 +	int result = diff_get_patch_id(options, sha1, diff_header_only);
  
  	for (i = 0; i < q->nr; i++)
  		diff_free_filepair(q->queue[i]);
@@@ -4844,7 -4838,7 +4866,7 @@@ static int diff_filespec_check_stat_unm
  	 */
  	if (!DIFF_FILE_VALID(p->one) || /* (1) */
  	    !DIFF_FILE_VALID(p->two) ||
 -	    (p->one->sha1_valid && p->two->sha1_valid) ||
 +	    (p->one->oid_valid && p->two->oid_valid) ||
  	    (p->one->mode != p->two->mode) ||
  	    diff_populate_filespec(p->one, CHECK_SIZE_ONLY) ||
  	    diff_populate_filespec(p->two, CHECK_SIZE_ONLY) ||
@@@ -5140,9 -5134,8 +5162,9 @@@ size_t fill_textconv(struct userdiff_dr
  	if (!driver->textconv)
  		die("BUG: fill_textconv called with non-textconv driver");
  
 -	if (driver->textconv_cache && df->sha1_valid) {
 -		*outbuf = notes_cache_get(driver->textconv_cache, df->sha1,
 +	if (driver->textconv_cache && df->oid_valid) {
 +		*outbuf = notes_cache_get(driver->textconv_cache,
 +					  df->oid.hash,
  					  &size);
  		if (*outbuf)
  			return size;
@@@ -5152,9 -5145,9 +5174,9 @@@
  	if (!*outbuf)
  		die("unable to read files to diff");
  
 -	if (driver->textconv_cache && df->sha1_valid) {
 +	if (driver->textconv_cache && df->oid_valid) {
  		/* ignore errors, as we might be in a readonly repository */
 -		notes_cache_put(driver->textconv_cache, df->sha1, *outbuf,
 +		notes_cache_put(driver->textconv_cache, df->oid.hash, *outbuf,
  				size);
  		/*
  		 * we could save up changes and flush them all at the end,
diff --combined diff.h
index ec76a90522,ab55a9920e..25ae60d5ff
--- a/diff.h
+++ b/diff.h
@@@ -83,6 -83,7 +83,6 @@@ typedef struct strbuf *(*diff_prefix_fn
  #define DIFF_OPT_DIRSTAT_BY_FILE     (1 << 20)
  #define DIFF_OPT_ALLOW_TEXTCONV      (1 << 21)
  #define DIFF_OPT_DIFF_FROM_CONTENTS  (1 << 22)
 -#define DIFF_OPT_SUBMODULE_LOG       (1 << 23)
  #define DIFF_OPT_DIRTY_SUBMODULES    (1 << 24)
  #define DIFF_OPT_IGNORE_UNTRACKED_IN_SUBMODULES (1 << 25)
  #define DIFF_OPT_IGNORE_DIRTY_SUBMODULES (1 << 26)
@@@ -109,19 -110,11 +109,19 @@@ enum diff_words_type 
  	DIFF_WORDS_COLOR
  };
  
 +enum diff_submodule_format {
 +	DIFF_SUBMODULE_SHORT = 0,
 +	DIFF_SUBMODULE_LOG,
 +	DIFF_SUBMODULE_INLINE_DIFF
 +};
 +
  struct diff_options {
  	const char *orderfile;
  	const char *pickaxe;
  	const char *single_follow;
  	const char *a_prefix, *b_prefix;
 +	const char *line_prefix;
 +	size_t line_prefix_length;
  	unsigned flags;
  	unsigned touched_flags;
  
@@@ -162,7 -155,6 +162,7 @@@
  	int stat_count;
  	const char *word_regex;
  	enum diff_words_type word_diff;
 +	enum diff_submodule_format submodule_format;
  
  	/* this is set by diffcore for DIFF_FORMAT_PATCH */
  	int found_changes;
@@@ -182,6 -174,7 +182,6 @@@
  	diff_format_fn_t format_callback;
  	void *format_callback_data;
  	diff_prefix_fn_t output_prefix;
 -	int output_prefix_length;
  	void *output_prefix_data;
  
  	int diff_path_counter;
@@@ -273,6 -266,7 +273,7 @@@ extern int parse_long_opt(const char *o
  			 const char **optarg);
  
  extern int git_diff_basic_config(const char *var, const char *value, void *cb);
+ extern int git_diff_heuristic_config(const char *var, const char *value, void *cb);
  extern void init_diff_ui_defaults(void);
  extern int git_diff_ui_config(const char *var, const char *value, void *cb);
  extern void diff_setup(struct diff_options *);
@@@ -349,7 -343,7 +350,7 @@@ extern int run_diff_files(struct rev_in
  extern int run_diff_index(struct rev_info *revs, int cached);
  
  extern int do_diff_cache(const unsigned char *, struct diff_options *);
 -extern int diff_flush_patch_id(struct diff_options *, unsigned char *);
 +extern int diff_flush_patch_id(struct diff_options *, unsigned char *, int);
  
  extern int diff_result_code(struct diff_options *, int);
  
diff --combined parse-options-cb.c
index 9667bc75a0,9f2f9b3df6..b5d920914e
--- a/parse-options-cb.c
+++ b/parse-options-cb.c
@@@ -117,24 -117,19 +117,24 @@@ int parse_opt_tertiary(const struct opt
  	return 0;
  }
  
 -int parse_options_concat(struct option *dst, size_t dst_size, struct option *src)
 +struct option *parse_options_concat(struct option *a, struct option *b)
  {
 -	int i, j;
 -
 -	for (i = 0; i < dst_size; i++)
 -		if (dst[i].type == OPTION_END)
 -			break;
 -	for (j = 0; i < dst_size; i++, j++) {
 -		dst[i] = src[j];
 -		if (src[j].type == OPTION_END)
 -			return 0;
 -	}
 -	return -1;
 +	struct option *ret;
 +	size_t i, a_len = 0, b_len = 0;
 +
 +	for (i = 0; a[i].type != OPTION_END; i++)
 +		a_len++;
 +	for (i = 0; b[i].type != OPTION_END; i++)
 +		b_len++;
 +
 +	ALLOC_ARRAY(ret, st_add3(a_len, b_len, 1));
 +	for (i = 0; i < a_len; i++)
 +		ret[i] = a[i];
 +	for (i = 0; i < b_len; i++)
 +		ret[a_len + i] = b[i];
 +	ret[a_len + b_len] = b[b_len]; /* final OPTION_END */
 +
 +	return ret;
  }
  
  int parse_opt_string_list(const struct option *opt, const char *arg, int unset)
@@@ -158,6 -153,18 +158,18 @@@ int parse_opt_noop_cb(const struct opti
  	return 0;
  }
  
+ /**
+  * Report that the option is unknown, so that other code can handle
+  * it. This can be used as a callback together with
+  * OPTION_LOWLEVEL_CALLBACK to allow an option to be documented in the
+  * "-h" output even if it's not being handled directly by
+  * parse_options().
+  */
+ int parse_opt_unknown_cb(const struct option *opt, const char *arg, int unset)
+ {
+ 	return -2;
+ }
+ 
  /**
   * Recreates the command-line option in the strbuf.
   */
diff --combined parse-options.h
index 78f8384c56,b2cc6a5574..dcd8a0926c
--- a/parse-options.h
+++ b/parse-options.h
@@@ -215,7 -215,7 +215,7 @@@ extern int parse_options_step(struct pa
  
  extern int parse_options_end(struct parse_opt_ctx_t *ctx);
  
 -extern int parse_options_concat(struct option *dst, size_t, struct option *src);
 +extern struct option *parse_options_concat(struct option *a, struct option *b);
  
  /*----- some often used options -----*/
  extern int parse_opt_abbrev_cb(const struct option *, const char *, int);
@@@ -228,6 -228,7 +228,7 @@@ extern int parse_opt_commits(const stru
  extern int parse_opt_tertiary(const struct option *, const char *, int);
  extern int parse_opt_string_list(const struct option *, const char *, int);
  extern int parse_opt_noop_cb(const struct option *, const char *, int);
+ extern int parse_opt_unknown_cb(const struct option *, const char *, int);
  extern int parse_opt_passthru(const struct option *, const char *, int);
  extern int parse_opt_passthru_argv(const struct option *, const char *, int);