From: Junio C Hamano <gitster@pobox.com>
Date: Wed, 18 Oct 2017 01:19:08 +0000 (+0900)
Subject: Merge branch 'jk/ref-filter-colors-fix'
X-Git-Tag: v2.15.0-rc2~7
X-Git-Url: https://git.lorimer.id.au/gitweb.git/diff_plain/1c0b983a777bf283250aaf00c94a9a9d4bf8bc9c?ds=inline;hp=-c

Merge branch 'jk/ref-filter-colors-fix'

This is the "theoretically more correct" approach of simply
stepping back to the state before plumbing commands started paying
attention to "color.ui" configuration variable.

Let's run with this one.

* jk/ref-filter-colors-fix:
tag: respect color.ui config
Revert "color: check color.ui in git_default_config()"
Revert "t6006: drop "always" color config tests"
Revert "color: make "always" the same as "auto" in config"
---

1c0b983a777bf283250aaf00c94a9a9d4bf8bc9c
diff --combined Documentation/config.txt
index b53c994d0a,2271809d90..1ac0ae6adb
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@@ -776,12 -776,6 +776,12 @@@ core.commentChar:
  If set to "auto", `git-commit` would select a character that is not
  the beginning character of any line in existing commit messages.
  
 +core.filesRefLockTimeout::
 +	The length of time, in milliseconds, to retry when trying to
 +	lock an individual reference. Value 0 means not to retry at
 +	all; -1 means to try indefinitely. Default is 100 (i.e.,
 +	retry for 100ms).
 +
  core.packedRefsTimeout::
  	The length of time, in milliseconds, to retry when trying to
  	lock the `packed-refs` file. Value 0 means not to retry at
@@@ -1058,10 -1052,10 +1058,10 @@@ clean.requireForce:
  
  color.branch::
  	A boolean to enable/disable color in the output of
- 	linkgit:git-branch[1]. May be set to `false` (or `never`) to
- 	disable color entirely, `auto` (or `true` or `always`) in which
- 	case colors are used only when the output is to a terminal.  If
- 	unset, then the value of `color.ui` is used (`auto` by default).
+ 	linkgit:git-branch[1]. May be set to `always`,
+ 	`false` (or `never`) or `auto` (or `true`), in which case colors are used
+ 	only when the output is to a terminal. If unset, then the
+ 	value of `color.ui` is used (`auto` by default).
  
  color.branch.<slot>::
  	Use customized color for branch coloration. `<slot>` is one of
@@@ -1072,35 -1066,25 +1072,36 @@@
  
  color.diff::
  	Whether to use ANSI escape sequences to add color to patches.
- 	If this is set to `true` or `auto`, linkgit:git-diff[1],
+ 	If this is set to `always`, linkgit:git-diff[1],
  	linkgit:git-log[1], and linkgit:git-show[1] will use color
- 	when output is to the terminal. The value `always` is a
- 	historical synonym for `auto`.  If unset, then the value of
- 	`color.ui` is used (`auto` by default).
+ 	for all patches.  If it is set to `true` or `auto`, those
+ 	commands will only use color when output is to the terminal.
+ 	If unset, then the value of `color.ui` is used (`auto` by
+ 	default).
  +
  This does not affect linkgit:git-format-patch[1] or the
  'git-diff-{asterisk}' plumbing commands.  Can be overridden on the
  command line with the `--color[=<when>]` option.
  
 +diff.colorMoved::
 +	If set to either a valid `<mode>` or a true value, moved lines
 +	in a diff are colored differently, for details of valid modes
 +	see '--color-moved' in linkgit:git-diff[1]. If simply set to
 +	true the default color mode will be used. When set to false,
 +	moved lines are not colored.
 +
  color.diff.<slot>::
  	Use customized color for diff colorization.  `<slot>` specifies
  	which part of the patch to use the specified color, and is one
  	of `context` (context text - `plain` is a historical synonym),
  	`meta` (metainformation), `frag`
  	(hunk header), 'func' (function in hunk header), `old` (removed lines),
 -	`new` (added lines), `commit` (commit headers), or `whitespace`
 -	(highlighting whitespace errors).
 +	`new` (added lines), `commit` (commit headers), `whitespace`
 +	(highlighting whitespace errors), `oldMoved` (deleted lines),
 +	`newMoved` (added lines), `oldMovedDimmed`, `oldMovedAlternative`,
 +	`oldMovedAlternativeDimmed`, `newMovedDimmed`, `newMovedAlternative`
 +	and `newMovedAlternativeDimmed` (See the '<mode>'
 +	setting of '--color-moved' in linkgit:git-diff[1] for details).
  
  color.decorate.<slot>::
  	Use customized color for 'git log --decorate' output.  `<slot>` is one
@@@ -1140,12 -1124,12 +1141,12 @@@ color.grep.<slot>:
  --
  
  color.interactive::
- 	When set to `true` or `auto`, use colors for interactive prompts
+ 	When set to `always`, always use colors for interactive prompts
  	and displays (such as those used by "git-add --interactive" and
- 	"git-clean --interactive") when the output is to the terminal.
- 	When false (or `never`), never show colors. The value `always`
- 	is a historical synonym for `auto`.  If unset, then the value of
- 	`color.ui` is used (`auto` by default).
+ 	"git-clean --interactive"). When false (or `never`), never.
+ 	When set to `true` or `auto`, use colors only when the output is
+ 	to the terminal. If unset, then the value of `color.ui` is
+ 	used (`auto` by default).
  
  color.interactive.<slot>::
  	Use customized color for 'git add --interactive' and 'git clean
@@@ -1192,10 -1176,10 +1193,10 @@@ color.ui:
  	configuration to set a default for the `--color` option.  Set it
  	to `false` or `never` if you prefer Git commands not to use
  	color unless enabled explicitly with some other configuration
- 	or the `--color` option. Set it to `true` or `auto` to enable
- 	color when output is written to the terminal (this is also the
- 	default since Git 1.8.4). The value `always` is a historical
- 	synonym for `auto`.
+ 	or the `--color` option. Set it to `always` if you want all
+ 	output not intended for machine consumption to use color, to
+ 	`true` or `auto` (this is the default since Git 1.8.4) if you
+ 	want such output to use color when written to the terminal.
  
  column.ui::
  	Specify whether supported commands should output in columns.
@@@ -1569,13 -1553,11 +1570,13 @@@ gc.<pattern>.reflogExpireUnreachable:
  gc.rerereResolved::
  	Records of conflicted merge you resolved earlier are
  	kept for this many days when 'git rerere gc' is run.
 +	You can also use more human-readable "1.month.ago", etc.
  	The default is 60 days.  See linkgit:git-rerere[1].
  
  gc.rerereUnresolved::
  	Records of conflicted merge you have not resolved are
  	kept for this many days when 'git rerere gc' is run.
 +	You can also use more human-readable "1.month.ago", etc.
  	The default is 15 days.  See linkgit:git-rerere[1].
  
  gitcvs.commitMsgAnnotation::
@@@ -3084,14 -3066,10 +3085,14 @@@ submodule.<name>.url:
  	See linkgit:git-submodule[1] and linkgit:gitmodules[5] for details.
  
  submodule.<name>.update::
 -	The default update procedure for a submodule. This variable
 -	is populated by `git submodule init` from the
 -	linkgit:gitmodules[5] file. See description of 'update'
 -	command in linkgit:git-submodule[1].
 +	The method by which a submodule is updated by 'git submodule update',
 +	which is the only affected command, others such as
 +	'git checkout --recurse-submodules' are unaffected. It exists for
 +	historical reasons, when 'git submodule' was the only command to
 +	interact with submodules; settings like `submodule.active`
 +	and `pull.rebase` are more specific. It is populated by
 +	`git submodule init` from the linkgit:gitmodules[5] file.
 +	See description of 'update' command in linkgit:git-submodule[1].
  
  submodule.<name>.branch::
  	The remote branch name for a submodule, used by `git submodule
diff --combined builtin/branch.c
index b67593288c,1969c7116c..79dc9181fd
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@@ -28,7 -28,6 +28,7 @@@ static const char * const builtin_branc
  	N_("git branch [<options>] [-l] [-f] <branch-name> [<start-point>]"),
  	N_("git branch [<options>] [-r] (-d | -D) <branch-name>..."),
  	N_("git branch [<options>] (-m | -M) [<old-branch>] <new-branch>"),
 +	N_("git branch [<options>] (-c | -C) [<old-branch>] <new-branch>"),
  	N_("git branch [<options>] [-r | -a] [--points-at]"),
  	N_("git branch [<options>] [-r | -a] [--format]"),
  	NULL
@@@ -93,7 -92,7 +93,7 @@@ static int git_branch_config(const cha
  			return config_error_nonbool(var);
  		return color_parse(value, branch_colors[slot]);
  	}
- 	return git_default_config(var, value, cb);
+ 	return git_color_default_config(var, value, cb);
  }
  
  static const char *branch_get_color(enum color_branch ix)
@@@ -217,7 -216,7 +217,7 @@@ static int delete_branches(int argc, co
  		if (!head_rev)
  			die(_("Couldn't look up commit object for HEAD"));
  	}
 -	for (i = 0; i < argc; i++, strbuf_release(&bname)) {
 +	for (i = 0; i < argc; i++, strbuf_reset(&bname)) {
  		char *target = NULL;
  		int flags = 0;
  
@@@ -282,9 -281,8 +282,9 @@@
  	}
  
  	free(name);
 +	strbuf_release(&bname);
  
 -	return(ret);
 +	return ret;
  }
  
  static int calc_maxwidth(struct ref_array *refs, int remote_bonus)
@@@ -354,7 -352,7 +354,7 @@@ static char *build_format(struct ref_fi
  			strbuf_addf(&obname, "%%(objectname:short=%d)", filter->abbrev);
  
  		strbuf_addf(&local, "%%(align:%d,left)%%(refname:lstrip=2)%%(end)", maxwidth);
 -		strbuf_addf(&local, "%s", branch_get_color(BRANCH_COLOR_RESET));
 +		strbuf_addstr(&local, branch_get_color(BRANCH_COLOR_RESET));
  		strbuf_addf(&local, " %s ", obname.buf);
  
  		if (filter->verbose > 1)
@@@ -458,19 -456,15 +458,19 @@@ static void reject_rebase_or_bisect_bra
  	free_worktrees(worktrees);
  }
  
 -static void rename_branch(const char *oldname, const char *newname, int force)
 +static void copy_or_rename_branch(const char *oldname, const char *newname, int copy, int force)
  {
  	struct strbuf oldref = STRBUF_INIT, newref = STRBUF_INIT, logmsg = STRBUF_INIT;
  	struct strbuf oldsection = STRBUF_INIT, newsection = STRBUF_INIT;
  	int recovery = 0;
  	int clobber_head_ok;
  
 -	if (!oldname)
 -		die(_("cannot rename the current branch while not on any."));
 +	if (!oldname) {
 +		if (copy)
 +			die(_("cannot copy the current branch while not on any."));
 +		else
 +			die(_("cannot rename the current branch while not on any."));
 +	}
  
  	if (strbuf_check_branch_ref(&oldref, oldname)) {
  		/*
@@@ -493,29 -487,16 +493,29 @@@
  
  	reject_rebase_or_bisect_branch(oldref.buf);
  
 -	strbuf_addf(&logmsg, "Branch: renamed %s to %s",
 -		 oldref.buf, newref.buf);
 +	if (copy)
 +		strbuf_addf(&logmsg, "Branch: copied %s to %s",
 +			    oldref.buf, newref.buf);
 +	else
 +		strbuf_addf(&logmsg, "Branch: renamed %s to %s",
 +			    oldref.buf, newref.buf);
  
 -	if (rename_ref(oldref.buf, newref.buf, logmsg.buf))
 +	if (!copy && rename_ref(oldref.buf, newref.buf, logmsg.buf))
  		die(_("Branch rename failed"));
 +	if (copy && copy_existing_ref(oldref.buf, newref.buf, logmsg.buf))
 +		die(_("Branch copy failed"));
  
 -	if (recovery)
 -		warning(_("Renamed a misnamed branch '%s' away"), oldref.buf + 11);
 +	if (recovery) {
 +		if (copy)
 +			warning(_("Copied a misnamed branch '%s' away"),
 +				oldref.buf + 11);
 +		else
 +			warning(_("Renamed a misnamed branch '%s' away"),
 +				oldref.buf + 11);
 +	}
  
 -	if (replace_each_worktree_head_symref(oldref.buf, newref.buf, logmsg.buf))
 +	if (!copy &&
 +	    replace_each_worktree_head_symref(oldref.buf, newref.buf, logmsg.buf))
  		die(_("Branch renamed to %s, but HEAD is not updated!"), newname);
  
  	strbuf_release(&logmsg);
@@@ -524,10 -505,8 +524,10 @@@
  	strbuf_release(&oldref);
  	strbuf_addf(&newsection, "branch.%s", newref.buf + 11);
  	strbuf_release(&newref);
 -	if (git_config_rename_section(oldsection.buf, newsection.buf) < 0)
 +	if (!copy && git_config_rename_section(oldsection.buf, newsection.buf) < 0)
  		die(_("Branch is renamed, but update of config-file failed"));
 +	if (copy && strcmp(oldname, newname) && git_config_copy_section(oldsection.buf, newsection.buf) < 0)
 +		die(_("Branch is copied, but update of config-file failed"));
  	strbuf_release(&oldsection);
  	strbuf_release(&newsection);
  }
@@@ -565,7 -544,7 +565,7 @@@ static int edit_branch_description(cons
  
  int cmd_branch(int argc, const char **argv, const char *prefix)
  {
 -	int delete = 0, rename = 0, force = 0, list = 0;
 +	int delete = 0, rename = 0, copy = 0, force = 0, list = 0;
  	int reflog = 0, edit_description = 0;
  	int quiet = 0, unset_upstream = 0;
  	const char *new_upstream = NULL;
@@@ -582,8 -561,8 +582,8 @@@
  		OPT__QUIET(&quiet, N_("suppress informational messages")),
  		OPT_SET_INT('t', "track",  &track, N_("set up tracking mode (see git-pull(1))"),
  			BRANCH_TRACK_EXPLICIT),
 -		OPT_SET_INT( 0, "set-upstream",  &track, N_("change upstream info"),
 -			BRANCH_TRACK_OVERRIDE),
 +		{ OPTION_SET_INT, 0, "set-upstream", &track, NULL, N_("do not use"),
 +			PARSE_OPT_NOARG | PARSE_OPT_HIDDEN, NULL, BRANCH_TRACK_OVERRIDE },
  		OPT_STRING('u', "set-upstream-to", &new_upstream, N_("upstream"), N_("change the upstream info")),
  		OPT_BOOL(0, "unset-upstream", &unset_upstream, N_("Unset the upstream info")),
  		OPT__COLOR(&branch_use_color, N_("use colored output")),
@@@ -602,8 -581,6 +602,8 @@@
  		OPT_BIT('D', NULL, &delete, N_("delete branch (even if not merged)"), 2),
  		OPT_BIT('m', "move", &rename, N_("move/rename a branch and its reflog"), 1),
  		OPT_BIT('M', NULL, &rename, N_("move/rename a branch, even if target exists"), 2),
 +		OPT_BIT('c', "copy", &copy, N_("copy a branch and its reflog"), 1),
 +		OPT_BIT('C', NULL, &copy, N_("copy a branch, even if target exists"), 2),
  		OPT_BOOL(0, "list", &list, N_("list branch names")),
  		OPT_BOOL('l', "create-reflog", &reflog, N_("create the branch's reflog")),
  		OPT_BOOL(0, "edit-description", &edit_description,
@@@ -647,14 -624,14 +647,14 @@@
  	argc = parse_options(argc, argv, prefix, options, builtin_branch_usage,
  			     0);
  
 -	if (!delete && !rename && !edit_description && !new_upstream && !unset_upstream && argc == 0)
 +	if (!delete && !rename && !copy && !edit_description && !new_upstream && !unset_upstream && argc == 0)
  		list = 1;
  
  	if (filter.with_commit || filter.merge != REF_FILTER_MERGED_NONE || filter.points_at.nr ||
  	    filter.no_commit)
  		list = 1;
  
 -	if (!!delete + !!rename + !!new_upstream +
 +	if (!!delete + !!rename + !!copy + !!new_upstream +
  	    list + unset_upstream > 1)
  		usage_with_options(builtin_branch_usage, options);
  
@@@ -672,7 -649,6 +672,7 @@@
  	if (force) {
  		delete *= 2;
  		rename *= 2;
 +		copy *= 2;
  	}
  
  	if (delete) {
@@@ -727,29 -703,20 +727,29 @@@
  
  		if (edit_branch_description(branch_name))
  			return 1;
 +	} else if (copy) {
 +		if (!argc)
 +			die(_("branch name required"));
 +		else if (argc == 1)
 +			copy_or_rename_branch(head, argv[0], 1, copy > 1);
 +		else if (argc == 2)
 +			copy_or_rename_branch(argv[0], argv[1], 1, copy > 1);
 +		else
 +			die(_("too many branches for a copy operation"));
  	} else if (rename) {
  		if (!argc)
  			die(_("branch name required"));
  		else if (argc == 1)
 -			rename_branch(head, argv[0], rename > 1);
 +			copy_or_rename_branch(head, argv[0], 0, rename > 1);
  		else if (argc == 2)
 -			rename_branch(argv[0], argv[1], rename > 1);
 +			copy_or_rename_branch(argv[0], argv[1], 0, rename > 1);
  		else
 -			die(_("too many branches for a rename operation"));
 +			die(_("too many arguments for a rename operation"));
  	} else if (new_upstream) {
  		struct branch *branch = branch_get(argv[0]);
  
  		if (argc > 1)
 -			die(_("too many branches to set new upstream"));
 +			die(_("too many arguments to set new upstream"));
  
  		if (!branch) {
  			if (!argc || !strcmp(argv[0], "HEAD"))
@@@ -772,7 -739,7 +772,7 @@@
  		struct strbuf buf = STRBUF_INIT;
  
  		if (argc > 1)
 -			die(_("too many branches to unset upstream"));
 +			die(_("too many arguments to unset upstream"));
  
  		if (!branch) {
  			if (!argc || !strcmp(argv[0], "HEAD"))
@@@ -792,6 -759,8 +792,6 @@@
  		strbuf_release(&buf);
  	} else if (argc > 0 && argc <= 2) {
  		struct branch *branch = branch_get(argv[0]);
 -		int branch_existed = 0, remote_tracking = 0;
 -		struct strbuf buf = STRBUF_INIT;
  
  		if (!strcmp(argv[0], "HEAD"))
  			die(_("it does not make sense to create 'HEAD' manually"));
@@@ -803,11 -772,28 +803,11 @@@
  			die(_("-a and -r options to 'git branch' do not make sense with a branch name"));
  
  		if (track == BRANCH_TRACK_OVERRIDE)
 -			fprintf(stderr, _("The --set-upstream flag is deprecated and will be removed. Consider using --track or --set-upstream-to\n"));
 -
 -		strbuf_addf(&buf, "refs/remotes/%s", branch->name);
 -		remote_tracking = ref_exists(buf.buf);
 -		strbuf_release(&buf);
 +			die(_("the '--set-upstream' option is no longer supported. Please use '--track' or '--set-upstream-to' instead."));
  
 -		branch_existed = ref_exists(branch->refname);
  		create_branch(argv[0], (argc == 2) ? argv[1] : head,
  			      force, reflog, 0, quiet, track);
  
 -		/*
 -		 * We only show the instructions if the user gave us
 -		 * one branch which doesn't exist locally, but is the
 -		 * name of a remote-tracking branch.
 -		 */
 -		if (argc == 1 && track == BRANCH_TRACK_OVERRIDE &&
 -		    !branch_existed && remote_tracking) {
 -			fprintf(stderr, _("\nIf you wanted to make '%s' track '%s', do this:\n\n"), head, branch->name);
 -			fprintf(stderr, "    git branch -d %s\n", branch->name);
 -			fprintf(stderr, "    git branch --set-upstream-to %s\n", branch->name);
 -		}
 -
  	} else
  		usage_with_options(builtin_branch_usage, options);
  
diff --combined builtin/clean.c
index 733b6d3745,057fc97fe4..189e20628c
--- a/builtin/clean.c
+++ b/builtin/clean.c
@@@ -33,6 -33,15 +33,6 @@@ static const char *msg_skip_git_dir = N
  static const char *msg_would_skip_git_dir = N_("Would skip repository %s\n");
  static const char *msg_warn_remove_failed = N_("failed to remove %s");
  
 -static int clean_use_color = -1;
 -static char clean_colors[][COLOR_MAXLEN] = {
 -	GIT_COLOR_RESET,
 -	GIT_COLOR_NORMAL,	/* PLAIN */
 -	GIT_COLOR_BOLD_BLUE,	/* PROMPT */
 -	GIT_COLOR_BOLD,		/* HEADER */
 -	GIT_COLOR_BOLD_RED,	/* HELP */
 -	GIT_COLOR_BOLD_RED,	/* ERROR */
 -};
  enum color_clean {
  	CLEAN_COLOR_RESET = 0,
  	CLEAN_COLOR_PLAIN = 1,
@@@ -42,16 -51,6 +42,16 @@@
  	CLEAN_COLOR_ERROR = 5
  };
  
 +static int clean_use_color = -1;
 +static char clean_colors[][COLOR_MAXLEN] = {
 +	[CLEAN_COLOR_ERROR] = GIT_COLOR_BOLD_RED,
 +	[CLEAN_COLOR_HEADER] = GIT_COLOR_BOLD,
 +	[CLEAN_COLOR_HELP] = GIT_COLOR_BOLD_RED,
 +	[CLEAN_COLOR_PLAIN] = GIT_COLOR_NORMAL,
 +	[CLEAN_COLOR_PROMPT] = GIT_COLOR_BOLD_BLUE,
 +	[CLEAN_COLOR_RESET] = GIT_COLOR_RESET,
 +};
 +
  #define MENU_OPTS_SINGLETON		01
  #define MENU_OPTS_IMMEDIATE		02
  #define MENU_OPTS_LIST_ONLY		04
@@@ -126,7 -125,8 +126,8 @@@ static int git_clean_config(const char 
  		return 0;
  	}
  
- 	return git_default_config(var, value, cb);
+ 	/* inspect the color.ui config variable and others */
+ 	return git_color_default_config(var, value, cb);
  }
  
  static const char *clean_get_color(enum color_clean ix)
@@@ -167,7 -167,7 +168,7 @@@ static int remove_dirs(struct strbuf *p
  		}
  
  		*dir_gone = 0;
 -		return 0;
 +		goto out;
  	}
  
  	dir = opendir(path->buf);
@@@ -181,8 -181,7 +182,8 @@@
  			warning_errno(_(msg_warn_remove_failed), quoted.buf);
  			*dir_gone = 0;
  		}
 -		return res;
 +		ret = res;
 +		goto out;
  	}
  
  	strbuf_complete(path, '/');
@@@ -250,8 -249,6 +251,8 @@@
  		for (i = 0; i < dels.nr; i++)
  			printf(dry_run ?  _(msg_would_remove) : _(msg_remove), dels.items[i].string);
  	}
 +out:
 +	strbuf_release(&quoted);
  	string_list_clear(&dels, 0);
  	return ret;
  }
diff --combined builtin/grep.c
index 19e23946ac,7e79eb1a75..2d65f27d01
--- a/builtin/grep.c
+++ b/builtin/grep.c
@@@ -28,7 -28,13 +28,7 @@@ static char const * const grep_usage[] 
  	NULL
  };
  
 -static const char *super_prefix;
  static int recurse_submodules;
 -static struct argv_array submodule_options = ARGV_ARRAY_INIT;
 -static const char *parent_basename;
 -
 -static int grep_submodule_launch(struct grep_opt *opt,
 -				 const struct grep_source *gs);
  
  #define GREP_NUM_THREADS_DEFAULT 8
  static int num_threads;
@@@ -180,7 -186,10 +180,7 @@@ static void *run(void *arg
  			break;
  
  		opt->output_priv = w;
 -		if (w->source.type == GREP_SOURCE_SUBMODULE)
 -			hit |= grep_submodule_launch(opt, &w->source);
 -		else
 -			hit |= grep_source(opt, &w->source);
 +		hit |= grep_source(opt, &w->source);
  		grep_source_clear_data(&w->source);
  		work_done(w);
  	}
@@@ -275,7 -284,7 +275,7 @@@ static int wait_all(void
  static int grep_cmd_config(const char *var, const char *value, void *cb)
  {
  	int st = grep_config(var, value, cb);
- 	if (git_default_config(var, value, cb) < 0)
+ 	if (git_color_default_config(var, value, cb) < 0)
  		st = -1;
  
  	if (!strcmp(var, "grep.threads")) {
@@@ -318,13 -327,21 +318,13 @@@ static int grep_oid(struct grep_opt *op
  {
  	struct strbuf pathbuf = STRBUF_INIT;
  
 -	if (super_prefix) {
 -		strbuf_add(&pathbuf, filename, tree_name_len);
 -		strbuf_addstr(&pathbuf, super_prefix);
 -		strbuf_addstr(&pathbuf, filename + tree_name_len);
 +	if (opt->relative && opt->prefix_length) {
 +		quote_path_relative(filename + tree_name_len, opt->prefix, &pathbuf);
 +		strbuf_insert(&pathbuf, 0, filename, tree_name_len);
  	} else {
  		strbuf_addstr(&pathbuf, filename);
  	}
  
 -	if (opt->relative && opt->prefix_length) {
 -		char *name = strbuf_detach(&pathbuf, NULL);
 -		quote_path_relative(name + tree_name_len, opt->prefix, &pathbuf);
 -		strbuf_insert(&pathbuf, 0, name, tree_name_len);
 -		free(name);
 -	}
 -
  #ifndef NO_PTHREADS
  	if (num_threads) {
  		add_work(opt, GREP_SOURCE_OID, pathbuf.buf, path, oid);
@@@ -349,10 -366,15 +349,10 @@@ static int grep_file(struct grep_opt *o
  {
  	struct strbuf buf = STRBUF_INIT;
  
 -	if (super_prefix)
 -		strbuf_addstr(&buf, super_prefix);
 -	strbuf_addstr(&buf, filename);
 -
 -	if (opt->relative && opt->prefix_length) {
 -		char *name = strbuf_detach(&buf, NULL);
 -		quote_path_relative(name, opt->prefix, &buf);
 -		free(name);
 -	}
 +	if (opt->relative && opt->prefix_length)
 +		quote_path_relative(filename, opt->prefix, &buf);
 +	else
 +		strbuf_addstr(&buf, filename);
  
  #ifndef NO_PTHREADS
  	if (num_threads) {
@@@ -399,89 -421,284 +399,89 @@@ static void run_pager(struct grep_opt *
  		exit(status);
  }
  
 -static void compile_submodule_options(const struct grep_opt *opt,
 -				      const char **argv,
 -				      int cached, int untracked,
 -				      int opt_exclude, int use_index,
 -				      int pattern_type_arg)
 -{
 -	struct grep_pat *pattern;
 -
 -	if (recurse_submodules)
 -		argv_array_push(&submodule_options, "--recurse-submodules");
 -
 -	if (cached)
 -		argv_array_push(&submodule_options, "--cached");
 -	if (!use_index)
 -		argv_array_push(&submodule_options, "--no-index");
 -	if (untracked)
 -		argv_array_push(&submodule_options, "--untracked");
 -	if (opt_exclude > 0)
 -		argv_array_push(&submodule_options, "--exclude-standard");
 -
 -	if (opt->invert)
 -		argv_array_push(&submodule_options, "-v");
 -	if (opt->ignore_case)
 -		argv_array_push(&submodule_options, "-i");
 -	if (opt->word_regexp)
 -		argv_array_push(&submodule_options, "-w");
 -	switch (opt->binary) {
 -	case GREP_BINARY_NOMATCH:
 -		argv_array_push(&submodule_options, "-I");
 -		break;
 -	case GREP_BINARY_TEXT:
 -		argv_array_push(&submodule_options, "-a");
 -		break;
 -	default:
 -		break;
 -	}
 -	if (opt->allow_textconv)
 -		argv_array_push(&submodule_options, "--textconv");
 -	if (opt->max_depth != -1)
 -		argv_array_pushf(&submodule_options, "--max-depth=%d",
 -				 opt->max_depth);
 -	if (opt->linenum)
 -		argv_array_push(&submodule_options, "-n");
 -	if (!opt->pathname)
 -		argv_array_push(&submodule_options, "-h");
 -	if (!opt->relative)
 -		argv_array_push(&submodule_options, "--full-name");
 -	if (opt->name_only)
 -		argv_array_push(&submodule_options, "-l");
 -	if (opt->unmatch_name_only)
 -		argv_array_push(&submodule_options, "-L");
 -	if (opt->null_following_name)
 -		argv_array_push(&submodule_options, "-z");
 -	if (opt->count)
 -		argv_array_push(&submodule_options, "-c");
 -	if (opt->file_break)
 -		argv_array_push(&submodule_options, "--break");
 -	if (opt->heading)
 -		argv_array_push(&submodule_options, "--heading");
 -	if (opt->pre_context)
 -		argv_array_pushf(&submodule_options, "--before-context=%d",
 -				 opt->pre_context);
 -	if (opt->post_context)
 -		argv_array_pushf(&submodule_options, "--after-context=%d",
 -				 opt->post_context);
 -	if (opt->funcname)
 -		argv_array_push(&submodule_options, "-p");
 -	if (opt->funcbody)
 -		argv_array_push(&submodule_options, "-W");
 -	if (opt->all_match)
 -		argv_array_push(&submodule_options, "--all-match");
 -	if (opt->debug)
 -		argv_array_push(&submodule_options, "--debug");
 -	if (opt->status_only)
 -		argv_array_push(&submodule_options, "-q");
 -
 -	switch (pattern_type_arg) {
 -	case GREP_PATTERN_TYPE_BRE:
 -		argv_array_push(&submodule_options, "-G");
 -		break;
 -	case GREP_PATTERN_TYPE_ERE:
 -		argv_array_push(&submodule_options, "-E");
 -		break;
 -	case GREP_PATTERN_TYPE_FIXED:
 -		argv_array_push(&submodule_options, "-F");
 -		break;
 -	case GREP_PATTERN_TYPE_PCRE:
 -		argv_array_push(&submodule_options, "-P");
 -		break;
 -	case GREP_PATTERN_TYPE_UNSPECIFIED:
 -		break;
 -	default:
 -		die("BUG: Added a new grep pattern type without updating switch statement");
 -	}
 -
 -	for (pattern = opt->pattern_list; pattern != NULL;
 -	     pattern = pattern->next) {
 -		switch (pattern->token) {
 -		case GREP_PATTERN:
 -			argv_array_pushf(&submodule_options, "-e%s",
 -					 pattern->pattern);
 -			break;
 -		case GREP_AND:
 -		case GREP_OPEN_PAREN:
 -		case GREP_CLOSE_PAREN:
 -		case GREP_NOT:
 -		case GREP_OR:
 -			argv_array_push(&submodule_options, pattern->pattern);
 -			break;
 -		/* BODY and HEAD are not used by git-grep */
 -		case GREP_PATTERN_BODY:
 -		case GREP_PATTERN_HEAD:
 -			break;
 -		}
 -	}
 -
 -	/*
 -	 * Limit number of threads for child process to use.
 -	 * This is to prevent potential fork-bomb behavior of git-grep as each
 -	 * submodule process has its own thread pool.
 -	 */
 -	argv_array_pushf(&submodule_options, "--threads=%d",
 -			 DIV_ROUND_UP(num_threads, 2));
 -
 -	/* Add Pathspecs */
 -	argv_array_push(&submodule_options, "--");
 -	for (; *argv; argv++)
 -		argv_array_push(&submodule_options, *argv);
 -}
 +static int grep_cache(struct grep_opt *opt, struct repository *repo,
 +		      const struct pathspec *pathspec, int cached);
 +static int grep_tree(struct grep_opt *opt, const struct pathspec *pathspec,
 +		     struct tree_desc *tree, struct strbuf *base, int tn_len,
 +		     int check_attr, struct repository *repo);
  
 -/*
 - * Launch child process to grep contents of a submodule
 - */
 -static int grep_submodule_launch(struct grep_opt *opt,
 -				 const struct grep_source *gs)
 +static int grep_submodule(struct grep_opt *opt, struct repository *superproject,
 +			  const struct pathspec *pathspec,
 +			  const struct object_id *oid,
 +			  const char *filename, const char *path)
  {
 -	struct child_process cp = CHILD_PROCESS_INIT;
 -	int status, i;
 -	const char *end_of_base;
 -	const char *name;
 -	struct strbuf child_output = STRBUF_INIT;
 -
 -	end_of_base = strchr(gs->name, ':');
 -	if (gs->identifier && end_of_base)
 -		name = end_of_base + 1;
 -	else
 -		name = gs->name;
 +	struct repository submodule;
 +	int hit;
  
 -	prepare_submodule_repo_env(&cp.env_array);
 -	argv_array_push(&cp.env_array, GIT_DIR_ENVIRONMENT);
 +	if (!is_submodule_active(superproject, path))
 +		return 0;
  
 -	if (opt->relative && opt->prefix_length)
 -		argv_array_pushf(&cp.env_array, "%s=%s",
 -				 GIT_TOPLEVEL_PREFIX_ENVIRONMENT,
 -				 opt->prefix);
 +	if (repo_submodule_init(&submodule, superproject, path))
 +		return 0;
  
 -	/* Add super prefix */
 -	argv_array_pushf(&cp.args, "--super-prefix=%s%s/",
 -			 super_prefix ? super_prefix : "",
 -			 name);
 -	argv_array_push(&cp.args, "grep");
 +	repo_read_gitmodules(&submodule);
  
  	/*
 -	 * Add basename of parent project
 -	 * When performing grep on a tree object the filename is prefixed
 -	 * with the object's name: 'tree-name:filename'.  In order to
 -	 * provide uniformity of output we want to pass the name of the
 -	 * parent project's object name to the submodule so the submodule can
 -	 * prefix its output with the parent's name and not its own OID.
 +	 * NEEDSWORK: This adds the submodule's object directory to the list of
 +	 * alternates for the single in-memory object store.  This has some bad
 +	 * consequences for memory (processed objects will never be freed) and
 +	 * performance (this increases the number of pack files git has to pay
 +	 * attention to, to the sum of the number of pack files in all the
 +	 * repositories processed so far).  This can be removed once the object
 +	 * store is no longer global and instead is a member of the repository
 +	 * object.
  	 */
 -	if (gs->identifier && end_of_base)
 -		argv_array_pushf(&cp.args, "--parent-basename=%.*s",
 -				 (int) (end_of_base - gs->name),
 -				 gs->name);
 +	add_to_alternates_memory(submodule.objectdir);
  
 -	/* Add options */
 -	for (i = 0; i < submodule_options.argc; i++) {
 -		/*
 -		 * If there is a tree identifier for the submodule, add the
 -		 * rev after adding the submodule options but before the
 -		 * pathspecs.  To do this we listen for the '--' and insert the
 -		 * oid before pushing the '--' onto the child process argv
 -		 * array.
 -		 */
 -		if (gs->identifier &&
 -		    !strcmp("--", submodule_options.argv[i])) {
 -			argv_array_push(&cp.args, oid_to_hex(gs->identifier));
 -		}
 +	if (oid) {
 +		struct object *object;
 +		struct tree_desc tree;
 +		void *data;
 +		unsigned long size;
 +		struct strbuf base = STRBUF_INIT;
  
 -		argv_array_push(&cp.args, submodule_options.argv[i]);
 -	}
 +		object = parse_object_or_die(oid, oid_to_hex(oid));
  
 -	cp.git_cmd = 1;
 -	cp.dir = gs->path;
 +		grep_read_lock();
 +		data = read_object_with_reference(object->oid.hash, tree_type,
 +						  &size, NULL);
 +		grep_read_unlock();
  
 -	/*
 -	 * Capture output to output buffer and check the return code from the
 -	 * child process.  A '0' indicates a hit, a '1' indicates no hit and
 -	 * anything else is an error.
 -	 */
 -	status = capture_command(&cp, &child_output, 0);
 -	if (status && (status != 1)) {
 -		/* flush the buffer */
 -		write_or_die(1, child_output.buf, child_output.len);
 -		die("process for submodule '%s' failed with exit code: %d",
 -		    gs->name, status);
 -	}
 +		if (!data)
 +			die(_("unable to read tree (%s)"), oid_to_hex(&object->oid));
  
 -	opt->output(opt, child_output.buf, child_output.len);
 -	strbuf_release(&child_output);
 -	/* invert the return code to make a hit equal to 1 */
 -	return !status;
 -}
 +		strbuf_addstr(&base, filename);
 +		strbuf_addch(&base, '/');
  
 -/*
 - * Prep grep structures for a submodule grep
 - * oid: the oid of the submodule or NULL if using the working tree
 - * filename: name of the submodule including tree name of parent
 - * path: location of the submodule
 - */
 -static int grep_submodule(struct grep_opt *opt, const struct object_id *oid,
 -			  const char *filename, const char *path)
 -{
 -	if (!is_submodule_active(the_repository, path))
 -		return 0;
 -	if (!is_submodule_populated_gently(path, NULL)) {
 -		/*
 -		 * If searching history, check for the presence of the
 -		 * submodule's gitdir before skipping the submodule.
 -		 */
 -		if (oid) {
 -			const struct submodule *sub =
 -					submodule_from_path(null_sha1, path);
 -			if (sub)
 -				path = git_path("modules/%s", sub->name);
 -
 -			if (!(is_directory(path) && is_git_directory(path)))
 -				return 0;
 -		} else {
 -			return 0;
 -		}
 +		init_tree_desc(&tree, data, size);
 +		hit = grep_tree(opt, pathspec, &tree, &base, base.len,
 +				object->type == OBJ_COMMIT, &submodule);
 +		strbuf_release(&base);
 +		free(data);
 +	} else {
 +		hit = grep_cache(opt, &submodule, pathspec, 1);
  	}
  
 -#ifndef NO_PTHREADS
 -	if (num_threads) {
 -		add_work(opt, GREP_SOURCE_SUBMODULE, filename, path, oid);
 -		return 0;
 -	} else
 -#endif
 -	{
 -		struct grep_source gs;
 -		int hit;
 -
 -		grep_source_init(&gs, GREP_SOURCE_SUBMODULE,
 -				 filename, path, oid);
 -		hit = grep_submodule_launch(opt, &gs);
 -
 -		grep_source_clear(&gs);
 -		return hit;
 -	}
 +	repo_clear(&submodule);
 +	return hit;
  }
  
 -static int grep_cache(struct grep_opt *opt, const struct pathspec *pathspec,
 -		      int cached)
 +static int grep_cache(struct grep_opt *opt, struct repository *repo,
 +		      const struct pathspec *pathspec, int cached)
  {
  	int hit = 0;
  	int nr;
  	struct strbuf name = STRBUF_INIT;
  	int name_base_len = 0;
 -	if (super_prefix) {
 -		name_base_len = strlen(super_prefix);
 -		strbuf_addstr(&name, super_prefix);
 +	if (repo->submodule_prefix) {
 +		name_base_len = strlen(repo->submodule_prefix);
 +		strbuf_addstr(&name, repo->submodule_prefix);
  	}
  
 -	read_cache();
 +	repo_read_index(repo);
  
 -	for (nr = 0; nr < active_nr; nr++) {
 -		const struct cache_entry *ce = active_cache[nr];
 +	for (nr = 0; nr < repo->index->cache_nr; nr++) {
 +		const struct cache_entry *ce = repo->index->cache[nr];
  		strbuf_setlen(&name, name_base_len);
  		strbuf_addstr(&name, ce->name);
  
@@@ -498,14 -715,14 +498,14 @@@
  			    ce_skip_worktree(ce)) {
  				if (ce_stage(ce) || ce_intent_to_add(ce))
  					continue;
 -				hit |= grep_oid(opt, &ce->oid, ce->name,
 -						 0, ce->name);
 +				hit |= grep_oid(opt, &ce->oid, name.buf,
 +						 0, name.buf);
  			} else {
 -				hit |= grep_file(opt, ce->name);
 +				hit |= grep_file(opt, name.buf);
  			}
  		} else if (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&
  			   submodule_path_match(pathspec, name.buf, NULL)) {
 -			hit |= grep_submodule(opt, NULL, ce->name, ce->name);
 +			hit |= grep_submodule(opt, repo, pathspec, NULL, ce->name, ce->name);
  		} else {
  			continue;
  		}
@@@ -513,8 -730,8 +513,8 @@@
  		if (ce_stage(ce)) {
  			do {
  				nr++;
 -			} while (nr < active_nr &&
 -				 !strcmp(ce->name, active_cache[nr]->name));
 +			} while (nr < repo->index->cache_nr &&
 +				 !strcmp(ce->name, repo->index->cache[nr]->name));
  			nr--; /* compensate for loop control */
  		}
  		if (hit && opt->status_only)
@@@ -527,7 -744,7 +527,7 @@@
  
  static int grep_tree(struct grep_opt *opt, const struct pathspec *pathspec,
  		     struct tree_desc *tree, struct strbuf *base, int tn_len,
 -		     int check_attr)
 +		     int check_attr, struct repository *repo)
  {
  	int hit = 0;
  	enum interesting match = entry_not_interesting;
@@@ -535,8 -752,8 +535,8 @@@
  	int old_baselen = base->len;
  	struct strbuf name = STRBUF_INIT;
  	int name_base_len = 0;
 -	if (super_prefix) {
 -		strbuf_addstr(&name, super_prefix);
 +	if (repo->submodule_prefix) {
 +		strbuf_addstr(&name, repo->submodule_prefix);
  		name_base_len = name.len;
  	}
  
@@@ -574,11 -791,11 +574,11 @@@
  			strbuf_addch(base, '/');
  			init_tree_desc(&sub, data, size);
  			hit |= grep_tree(opt, pathspec, &sub, base, tn_len,
 -					 check_attr);
 +					 check_attr, repo);
  			free(data);
  		} else if (recurse_submodules && S_ISGITLINK(entry.mode)) {
 -			hit |= grep_submodule(opt, entry.oid, base->buf,
 -					      base->buf + tn_len);
 +			hit |= grep_submodule(opt, repo, pathspec, entry.oid,
 +					      base->buf, base->buf + tn_len);
  		}
  
  		strbuf_setlen(base, old_baselen);
@@@ -592,8 -809,7 +592,8 @@@
  }
  
  static int grep_object(struct grep_opt *opt, const struct pathspec *pathspec,
 -		       struct object *obj, const char *name, const char *path)
 +		       struct object *obj, const char *name, const char *path,
 +		       struct repository *repo)
  {
  	if (obj->type == OBJ_BLOB)
  		return grep_oid(opt, &obj->oid, name, 0, path);
@@@ -612,6 -828,10 +612,6 @@@
  		if (!data)
  			die(_("unable to read tree (%s)"), oid_to_hex(&obj->oid));
  
 -		/* Use parent's name as base when recursing submodules */
 -		if (recurse_submodules && parent_basename)
 -			name = parent_basename;
 -
  		len = name ? strlen(name) : 0;
  		strbuf_init(&base, PATH_MAX + len + 1);
  		if (len) {
@@@ -620,7 -840,7 +620,7 @@@
  		}
  		init_tree_desc(&tree, data, size);
  		hit = grep_tree(opt, pathspec, &tree, &base, base.len,
 -				obj->type == OBJ_COMMIT);
 +				obj->type == OBJ_COMMIT, repo);
  		strbuf_release(&base);
  		free(data);
  		return hit;
@@@ -629,7 -849,6 +629,7 @@@
  }
  
  static int grep_objects(struct grep_opt *opt, const struct pathspec *pathspec,
 +			struct repository *repo,
  			const struct object_array *list)
  {
  	unsigned int i;
@@@ -643,10 -862,9 +643,10 @@@
  		/* load the gitmodules file for this rev */
  		if (recurse_submodules) {
  			submodule_free();
 -			gitmodules_config_sha1(real_obj->oid.hash);
 +			gitmodules_config_oid(&real_obj->oid);
  		}
 -		if (grep_object(opt, pathspec, real_obj, list->objects[i].name, list->objects[i].path)) {
 +		if (grep_object(opt, pathspec, real_obj, list->objects[i].name, list->objects[i].path,
 +				repo)) {
  			hit = 1;
  			if (opt->status_only)
  				break;
@@@ -787,6 -1005,9 +787,6 @@@ int cmd_grep(int argc, const char **arg
  			    N_("ignore files specified via '.gitignore'"), 1),
  		OPT_BOOL(0, "recurse-submodules", &recurse_submodules,
  			 N_("recursively search in each submodule")),
 -		OPT_STRING(0, "parent-basename", &parent_basename,
 -			   N_("basename"),
 -			   N_("prepend parent project's basename to output")),
  		OPT_GROUP(""),
  		OPT_BOOL('v', "invert-match", &opt.invert,
  			N_("show non-matching lines")),
@@@ -891,6 -1112,7 +891,6 @@@
  	init_grep_defaults();
  	git_config(grep_cmd_config, NULL);
  	grep_init(&opt, prefix);
 -	super_prefix = get_super_prefix();
  
  	/*
  	 * If there is no -- then the paths must exist in the working
@@@ -983,8 -1205,8 +983,8 @@@
  			break;
  		}
  
 -		if (get_sha1_with_context(arg, GET_SHA1_RECORD_PATH,
 -					  oid.hash, &oc)) {
 +		if (get_oid_with_context(arg, GET_OID_RECORD_PATH,
 +					 &oid, &oc)) {
  			if (seen_dashdash)
  				die(_("unable to resolve revision: %s"), arg);
  			break;
@@@ -1048,6 -1270,13 +1048,6 @@@
  	}
  #endif
  
 -	if (recurse_submodules) {
 -		gitmodules_config();
 -		compile_submodule_options(&opt, argv + i, cached, untracked,
 -					  opt_exclude, use_index,
 -					  pattern_type_arg);
 -	}
 -
  	if (show_in_pager && (cached || list.nr))
  		die(_("--open-files-in-pager only works on the worktree"));
  
@@@ -1089,12 -1318,11 +1089,12 @@@
  		if (!cached)
  			setup_work_tree();
  
 -		hit = grep_cache(&opt, &pathspec, cached);
 +		hit = grep_cache(&opt, the_repository, &pathspec, cached);
  	} else {
  		if (cached)
  			die(_("both --cached and trees are given."));
 -		hit = grep_objects(&opt, &pathspec, &list);
 +
 +		hit = grep_objects(&opt, &pathspec, the_repository, &list);
  	}
  
  	if (num_threads)
diff --combined builtin/show-branch.c
index 84547d6fba,7073a3eb97..6fa1f62a88
--- a/builtin/show-branch.c
+++ b/builtin/show-branch.c
@@@ -393,7 -393,7 +393,7 @@@ static int append_head_ref(const char *
  	/* If both heads/foo and tags/foo exists, get_sha1 would
  	 * get confused.
  	 */
 -	if (get_sha1(refname + ofs, tmp.hash) || oidcmp(&tmp, oid))
 +	if (get_oid(refname + ofs, &tmp) || oidcmp(&tmp, oid))
  		ofs = 5;
  	return append_ref(refname + ofs, oid, 0);
  }
@@@ -408,7 -408,7 +408,7 @@@ static int append_remote_ref(const cha
  	/* If both heads/foo and tags/foo exists, get_sha1 would
  	 * get confused.
  	 */
 -	if (get_sha1(refname + ofs, tmp.hash) || oidcmp(&tmp, oid))
 +	if (get_oid(refname + ofs, &tmp) || oidcmp(&tmp, oid))
  		ofs = 5;
  	return append_ref(refname + ofs, oid, 0);
  }
@@@ -514,7 -514,7 +514,7 @@@ static int show_independent(struct comm
  static void append_one_rev(const char *av)
  {
  	struct object_id revkey;
 -	if (!get_sha1(av, revkey.hash)) {
 +	if (!get_oid(av, &revkey)) {
  		append_ref(av, &revkey, 0);
  		return;
  	}
@@@ -554,7 -554,7 +554,7 @@@ static int git_show_branch_config(cons
  		return 0;
  	}
  
- 	return git_default_config(var, value, cb);
+ 	return git_color_default_config(var, value, cb);
  }
  
  static int omit_in_dense(struct commit *commit, struct commit **rev, int n)
@@@ -808,7 -808,7 +808,7 @@@ int cmd_show_branch(int ac, const char 
  			die(Q_("cannot handle more than %d rev.",
  			       "cannot handle more than %d revs.",
  			       MAX_REVS), MAX_REVS);
 -		if (get_sha1(ref_name[num_rev], revkey.hash))
 +		if (get_oid(ref_name[num_rev], &revkey))
  			die(_("'%s' is not a valid ref."), ref_name[num_rev]);
  		commit = lookup_commit_reference(&revkey);
  		if (!commit)
diff --combined builtin/tag.c
index 695cb0778e,00382a56f5..b38329b593
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@@ -113,7 -113,7 +113,7 @@@ static int verify_tag(const char *name
  	if (format->format)
  		flags = GPG_VERIFY_OMIT_STATUS;
  
 -	if (gpg_verify_tag(oid->hash, name, flags))
 +	if (gpg_verify_tag(oid, name, flags))
  		return -1;
  
  	if (format->format)
@@@ -158,7 -158,7 +158,7 @@@ static int git_tag_config(const char *v
  
  	if (starts_with(var, "column."))
  		return git_column_config(var, value, "tag", &colopts);
- 	return git_default_config(var, value, cb);
+ 	return git_color_default_config(var, value, cb);
  }
  
  static void write_tag_body(int fd, const struct object_id *oid)
@@@ -553,10 -553,9 +553,10 @@@ int cmd_tag(int argc, const char **argv
  	if (force && !is_null_oid(&prev) && oidcmp(&prev, &object))
  		printf(_("Updated tag '%s' (was %s)\n"), tag, find_unique_abbrev(prev.hash, DEFAULT_ABBREV));
  
 -	strbuf_release(&err);
 -	strbuf_release(&buf);
 -	strbuf_release(&ref);
 -	strbuf_release(&reflog_msg);
 +	UNLEAK(buf);
 +	UNLEAK(ref);
 +	UNLEAK(reflog_msg);
 +	UNLEAK(msg);
 +	UNLEAK(err);
  	return 0;
  }
diff --combined color.c
index 9c0dc82370,31b6207a00..9a9261ac16
--- a/color.c
+++ b/color.c
@@@ -308,7 -308,7 +308,7 @@@ int git_config_colorbool(const char *va
  		if (!strcasecmp(value, "never"))
  			return 0;
  		if (!strcasecmp(value, "always"))
- 			return var ? GIT_COLOR_AUTO : 1;
+ 			return 1;
  		if (!strcasecmp(value, "auto"))
  			return GIT_COLOR_AUTO;
  	}
@@@ -338,13 -338,6 +338,13 @@@ static int check_auto_color(void
  
  int want_color(int var)
  {
 +	/*
 +	 * NEEDSWORK: This function is sometimes used from multiple threads, and
 +	 * we end up using want_auto racily. That "should not matter" since
 +	 * we always write the same value, but it's still wrong. This function
 +	 * is listed in .tsan-suppressions for the time being.
 +	 */
 +
  	static int want_auto = -1;
  
  	if (var < 0)
@@@ -368,6 -361,14 +368,14 @@@ int git_color_config(const char *var, c
  	return 0;
  }
  
+ int git_color_default_config(const char *var, const char *value, void *cb)
+ {
+ 	if (git_color_config(var, value, cb) < 0)
+ 		return -1;
+ 
+ 	return git_default_config(var, value, cb);
+ }
+ 
  void color_print_strbuf(FILE *fp, const char *color, const struct strbuf *sb)
  {
  	if (*color)
diff --combined config.c
index 4831c12735,231f9a750b..adb7d7a3e5
--- a/config.c
+++ b/config.c
@@@ -16,7 -16,6 +16,6 @@@
  #include "string-list.h"
  #include "utf8.h"
  #include "dir.h"
- #include "color.h"
  
  struct config_source {
  	struct config_source *prev;
@@@ -929,7 -928,7 +928,7 @@@ ssize_t git_config_ssize_t(const char *
  	return ret;
  }
  
 -int git_parse_maybe_bool(const char *value)
 +static int git_parse_maybe_bool_text(const char *value)
  {
  	if (!value)
  		return 1;
@@@ -946,9 -945,9 +945,9 @@@
  	return -1;
  }
  
 -int git_config_maybe_bool(const char *name, const char *value)
 +int git_parse_maybe_bool(const char *value)
  {
 -	int v = git_parse_maybe_bool(value);
 +	int v = git_parse_maybe_bool_text(value);
  	if (0 <= v)
  		return v;
  	if (git_parse_int(value, &v))
@@@ -958,7 -957,7 +957,7 @@@
  
  int git_config_bool_or_int(const char *name, const char *value, int *is_bool)
  {
 -	int v = git_parse_maybe_bool(value);
 +	int v = git_parse_maybe_bool_text(value);
  	if (0 <= v) {
  		*is_bool = 1;
  		return v;
@@@ -1351,9 -1350,6 +1350,6 @@@ int git_default_config(const char *var
  	if (starts_with(var, "advice."))
  		return git_default_advice_config(var, value);
  
- 	if (git_color_config(var, value, dummy) < 0)
- 		return -1;
- 
  	if (!strcmp(var, "pager.color") || !strcmp(var, "color.pager")) {
  		pager_use_color = git_config_bool(var,value);
  		return 0;
@@@ -1464,9 -1460,9 +1460,9 @@@ int git_config_from_mem(config_fn_t fn
  	return do_config_from(&top, fn, data);
  }
  
 -int git_config_from_blob_sha1(config_fn_t fn,
 +int git_config_from_blob_oid(config_fn_t fn,
  			      const char *name,
 -			      const unsigned char *sha1,
 +			      const struct object_id *oid,
  			      void *data)
  {
  	enum object_type type;
@@@ -1474,7 -1470,7 +1470,7 @@@
  	unsigned long size;
  	int ret;
  
 -	buf = read_sha1_file(sha1, &type, &size);
 +	buf = read_sha1_file(oid->hash, &type, &size);
  	if (!buf)
  		return error("unable to load config blob object '%s'", name);
  	if (type != OBJ_BLOB) {
@@@ -1492,11 -1488,11 +1488,11 @@@ static int git_config_from_blob_ref(con
  				    const char *name,
  				    void *data)
  {
 -	unsigned char sha1[20];
 +	struct object_id oid;
  
 -	if (get_sha1(name, sha1) < 0)
 +	if (get_oid(name, &oid) < 0)
  		return error("unable to resolve config blob '%s'", name);
 -	return git_config_from_blob_sha1(fn, name, sha1, data);
 +	return git_config_from_blob_oid(fn, name, &oid, data);
  }
  
  const char *git_etc_gitconfig(void)
@@@ -1719,19 -1715,17 +1715,19 @@@ static int configset_add_value(struct c
  }
  
  static int config_set_element_cmp(const void *unused_cmp_data,
 -				  const struct config_set_element *e1,
 -				  const struct config_set_element *e2,
 +				  const void *entry,
 +				  const void *entry_or_key,
  				  const void *unused_keydata)
  {
 +	const struct config_set_element *e1 = entry;
 +	const struct config_set_element *e2 = entry_or_key;
 +
  	return strcmp(e1->key, e2->key);
  }
  
  void git_configset_init(struct config_set *cs)
  {
 -	hashmap_init(&cs->config_hash, (hashmap_cmp_fn)config_set_element_cmp,
 -		     NULL, 0);
 +	hashmap_init(&cs->config_hash, config_set_element_cmp, NULL, 0);
  	cs->hash_initialized = 1;
  	cs->list.nr = 0;
  	cs->list.alloc = 0;
@@@ -1852,7 -1846,7 +1848,7 @@@ int git_configset_get_maybe_bool(struc
  {
  	const char *value;
  	if (!git_configset_get_value(cs, key, &value)) {
 -		*dest = git_config_maybe_bool(key, value);
 +		*dest = git_parse_maybe_bool(value);
  		if (*dest == -1)
  			return -1;
  		return 0;
@@@ -2059,23 -2053,6 +2055,23 @@@ int git_config_get_pathname(const char 
  	return repo_config_get_pathname(the_repository, key, dest);
  }
  
 +/*
 + * Note: This function exists solely to maintain backward compatibility with
 + * 'fetch' and 'update_clone' storing configuration in '.gitmodules' and should
 + * NOT be used anywhere else.
 + *
 + * Runs the provided config function on the '.gitmodules' file found in the
 + * working directory.
 + */
 +void config_from_gitmodules(config_fn_t fn, void *data)
 +{
 +	if (the_repository->worktree) {
 +		char *file = repo_worktree_path(the_repository, GITMODULES_FILE);
 +		git_config_from_file(fn, file, data);
 +		free(file);
 +	}
 +}
 +
  int git_config_get_expiry(const char *key, const char **output)
  {
  	int ret = git_config_get_string_const(key, output);
@@@ -2089,28 -2066,6 +2085,28 @@@
  	return ret;
  }
  
 +int git_config_get_expiry_in_days(const char *key, timestamp_t *expiry, timestamp_t now)
 +{
 +	char *expiry_string;
 +	intmax_t days;
 +	timestamp_t when;
 +
 +	if (git_config_get_string(key, &expiry_string))
 +		return 1; /* no such thing */
 +
 +	if (git_parse_signed(expiry_string, &days, maximum_signed_value_of_type(int))) {
 +		const int scale = 86400;
 +		*expiry = now - days * scale;
 +		return 0;
 +	}
 +
 +	if (!parse_expiry_date(expiry_string, &when)) {
 +		*expiry = when;
 +		return 0;
 +	}
 +	return -1; /* thing exists but cannot be parsed */
 +}
 +
  int git_config_get_untracked_cache(void)
  {
  	int val = -1;
@@@ -2200,7 -2155,7 +2196,7 @@@ static struct 
  	size_t *offset;
  	unsigned int offset_alloc;
  	enum { START, SECTION_SEEN, SECTION_END_SEEN, KEY_SEEN } state;
 -	int seen;
 +	unsigned int seen;
  } store;
  
  static int matches(const char *key, const char *value)
@@@ -2292,10 -2247,10 +2288,10 @@@ static int write_error(const char *file
  	return 4;
  }
  
 -static int store_write_section(int fd, const char *key)
 +static struct strbuf store_create_section(const char *key)
  {
  	const char *dot;
 -	int i, success;
 +	int i;
  	struct strbuf sb = STRBUF_INIT;
  
  	dot = memchr(key, '.', store.baselen);
@@@ -2311,24 -2266,15 +2307,24 @@@
  		strbuf_addf(&sb, "[%.*s]\n", store.baselen, key);
  	}
  
 -	success = write_in_full(fd, sb.buf, sb.len) == sb.len;
 +	return sb;
 +}
 +
 +static ssize_t write_section(int fd, const char *key)
 +{
 +	struct strbuf sb = store_create_section(key);
 +	ssize_t ret;
 +
 +	ret = write_in_full(fd, sb.buf, sb.len) == sb.len;
  	strbuf_release(&sb);
  
 -	return success;
 +	return ret;
  }
  
 -static int store_write_pair(int fd, const char *key, const char *value)
 +static ssize_t write_pair(int fd, const char *key, const char *value)
  {
 -	int i, success;
 +	int i;
 +	ssize_t ret;
  	int length = strlen(key + store.baselen + 1);
  	const char *quote = "";
  	struct strbuf sb = STRBUF_INIT;
@@@ -2362,17 -2308,16 +2358,17 @@@
  		case '"':
  		case '\\':
  			strbuf_addch(&sb, '\\');
 +			/* fallthrough */
  		default:
  			strbuf_addch(&sb, value[i]);
  			break;
  		}
  	strbuf_addf(&sb, "%s\n", quote);
  
 -	success = write_in_full(fd, sb.buf, sb.len) == sb.len;
 +	ret = write_in_full(fd, sb.buf, sb.len);
  	strbuf_release(&sb);
  
 -	return success;
 +	return ret;
  }
  
  static ssize_t find_beginning_of_line(const char *contents, size_t size,
@@@ -2455,7 -2400,7 +2451,7 @@@ int git_config_set_multivar_in_file_gen
  {
  	int fd = -1, in_fd = -1;
  	int ret;
 -	struct lock_file *lock = NULL;
 +	struct lock_file lock = LOCK_INIT;
  	char *filename_buf = NULL;
  	char *contents = NULL;
  	size_t contents_sz;
@@@ -2474,7 -2419,8 +2470,7 @@@
  	 * The lock serves a purpose in addition to locking: the new
  	 * contents of .git/config will be written into it.
  	 */
 -	lock = xcalloc(1, sizeof(struct lock_file));
 -	fd = hold_lock_file_for_update(lock, config_filename, 0);
 +	fd = hold_lock_file_for_update(&lock, config_filename, 0);
  	if (fd < 0) {
  		error_errno("could not lock config file %s", config_filename);
  		free(store.key);
@@@ -2501,8 -2447,8 +2497,8 @@@
  		}
  
  		store.key = (char *)key;
 -		if (!store_write_section(fd, key) ||
 -		    !store_write_pair(fd, key, value))
 +		if (write_section(fd, key) < 0 ||
 +		    write_pair(fd, key, value) < 0)
  			goto write_err_out;
  	} else {
  		struct stat st;
@@@ -2587,8 -2533,8 +2583,8 @@@
  		close(in_fd);
  		in_fd = -1;
  
 -		if (chmod(get_lock_file_path(lock), st.st_mode & 07777) < 0) {
 -			error_errno("chmod on %s failed", get_lock_file_path(lock));
 +		if (chmod(get_lock_file_path(&lock), st.st_mode & 07777) < 0) {
 +			error_errno("chmod on %s failed", get_lock_file_path(&lock));
  			ret = CONFIG_NO_WRITE;
  			goto out_free;
  		}
@@@ -2612,10 -2558,11 +2608,10 @@@
  			/* write the first part of the config */
  			if (copy_end > copy_begin) {
  				if (write_in_full(fd, contents + copy_begin,
 -						  copy_end - copy_begin) <
 -				    copy_end - copy_begin)
 +						  copy_end - copy_begin) < 0)
  					goto write_err_out;
  				if (new_line &&
 -				    write_str_in_full(fd, "\n") != 1)
 +				    write_str_in_full(fd, "\n") < 0)
  					goto write_err_out;
  			}
  			copy_begin = store.offset[i];
@@@ -2624,36 -2571,46 +2620,36 @@@
  		/* write the pair (value == NULL means unset) */
  		if (value != NULL) {
  			if (store.state == START) {
 -				if (!store_write_section(fd, key))
 +				if (write_section(fd, key) < 0)
  					goto write_err_out;
  			}
 -			if (!store_write_pair(fd, key, value))
 +			if (write_pair(fd, key, value) < 0)
  				goto write_err_out;
  		}
  
  		/* write the rest of the config */
  		if (copy_begin < contents_sz)
  			if (write_in_full(fd, contents + copy_begin,
 -					  contents_sz - copy_begin) <
 -			    contents_sz - copy_begin)
 +					  contents_sz - copy_begin) < 0)
  				goto write_err_out;
  
  		munmap(contents, contents_sz);
  		contents = NULL;
  	}
  
 -	if (commit_lock_file(lock) < 0) {
 +	if (commit_lock_file(&lock) < 0) {
  		error_errno("could not write config file %s", config_filename);
  		ret = CONFIG_NO_WRITE;
 -		lock = NULL;
  		goto out_free;
  	}
  
 -	/*
 -	 * lock is committed, so don't try to roll it back below.
 -	 * NOTE: Since lockfile.c keeps a linked list of all created
 -	 * lock_file structures, it isn't safe to free(lock).  It's
 -	 * better to just leave it hanging around.
 -	 */
 -	lock = NULL;
  	ret = 0;
  
  	/* Invalidate the config cache */
  	git_config_clear();
  
  out_free:
 -	if (lock)
 -		rollback_lock_file(lock);
 +	rollback_lock_file(&lock);
  	free(filename_buf);
  	if (contents)
  		munmap(contents, contents_sz);
@@@ -2662,7 -2619,7 +2658,7 @@@
  	return ret;
  
  write_err_out:
 -	ret = write_error(get_lock_file_path(lock));
 +	ret = write_error(get_lock_file_path(&lock));
  	goto out_free;
  
  }
@@@ -2750,8 -2707,8 +2746,8 @@@ static int section_name_is_ok(const cha
  }
  
  /* if new_name == NULL, the section is removed instead */
 -int git_config_rename_section_in_file(const char *config_filename,
 -				      const char *old_name, const char *new_name)
 +static int git_config_copy_or_rename_section_in_file(const char *config_filename,
 +				      const char *old_name, const char *new_name, int copy)
  {
  	int ret = 0, remove = 0;
  	char *filename_buf = NULL;
@@@ -2760,7 -2717,6 +2756,7 @@@
  	char buf[1024];
  	FILE *config_file = NULL;
  	struct stat st;
 +	struct strbuf copystr = STRBUF_INIT;
  
  	if (new_name && !section_name_is_ok(new_name)) {
  		ret = error("invalid section name: %s", new_name);
@@@ -2799,30 -2755,12 +2795,30 @@@
  	while (fgets(buf, sizeof(buf), config_file)) {
  		int i;
  		int length;
 +		int is_section = 0;
  		char *output = buf;
  		for (i = 0; buf[i] && isspace(buf[i]); i++)
  			; /* do nothing */
  		if (buf[i] == '[') {
  			/* it's a section */
 -			int offset = section_name_match(&buf[i], old_name);
 +			int offset;
 +			is_section = 1;
 +
 +			/*
 +			 * When encountering a new section under -c we
 +			 * need to flush out any section we're already
 +			 * coping and begin anew. There might be
 +			 * multiple [branch "$name"] sections.
 +			 */
 +			if (copystr.len > 0) {
 +				if (write_in_full(out_fd, copystr.buf, copystr.len) != copystr.len) {
 +					ret = write_error(get_lock_file_path(lock));
 +					goto out;
 +				}
 +				strbuf_reset(&copystr);
 +			}
 +
 +			offset = section_name_match(&buf[i], old_name);
  			if (offset > 0) {
  				ret++;
  				if (new_name == NULL) {
@@@ -2830,29 -2768,25 +2826,29 @@@
  					continue;
  				}
  				store.baselen = strlen(new_name);
 -				if (!store_write_section(out_fd, new_name)) {
 -					ret = write_error(get_lock_file_path(lock));
 -					goto out;
 -				}
 -				/*
 -				 * We wrote out the new section, with
 -				 * a newline, now skip the old
 -				 * section's length
 -				 */
 -				output += offset + i;
 -				if (strlen(output) > 0) {
 +				if (!copy) {
 +					if (write_section(out_fd, new_name) < 0) {
 +						ret = write_error(get_lock_file_path(lock));
 +						goto out;
 +					}
  					/*
 -					 * More content means there's
 -					 * a declaration to put on the
 -					 * next line; indent with a
 -					 * tab
 +					 * We wrote out the new section, with
 +					 * a newline, now skip the old
 +					 * section's length
  					 */
 -					output -= 1;
 -					output[0] = '\t';
 +					output += offset + i;
 +					if (strlen(output) > 0) {
 +						/*
 +						 * More content means there's
 +						 * a declaration to put on the
 +						 * next line; indent with a
 +						 * tab
 +						 */
 +						output -= 1;
 +						output[0] = '\t';
 +					}
 +				} else {
 +					copystr = store_create_section(new_name);
  				}
  			}
  			remove = 0;
@@@ -2860,30 -2794,11 +2856,30 @@@
  		if (remove)
  			continue;
  		length = strlen(output);
 -		if (write_in_full(out_fd, output, length) != length) {
 +
 +		if (!is_section && copystr.len > 0) {
 +			strbuf_add(&copystr, output, length);
 +		}
 +
 +		if (write_in_full(out_fd, output, length) < 0) {
  			ret = write_error(get_lock_file_path(lock));
  			goto out;
  		}
  	}
 +
 +	/*
 +	 * Copy a trailing section at the end of the config, won't be
 +	 * flushed by the usual "flush because we have a new section
 +	 * logic in the loop above.
 +	 */
 +	if (copystr.len > 0) {
 +		if (write_in_full(out_fd, copystr.buf, copystr.len) != copystr.len) {
 +			ret = write_error(get_lock_file_path(lock));
 +			goto out;
 +		}
 +		strbuf_reset(&copystr);
 +	}
 +
  	fclose(config_file);
  	config_file = NULL;
  commit_and_out:
@@@ -2899,30 -2814,11 +2895,30 @@@ out_no_rollback
  	return ret;
  }
  
 +int git_config_rename_section_in_file(const char *config_filename,
 +				      const char *old_name, const char *new_name)
 +{
 +	return git_config_copy_or_rename_section_in_file(config_filename,
 +					 old_name, new_name, 0);
 +}
 +
  int git_config_rename_section(const char *old_name, const char *new_name)
  {
  	return git_config_rename_section_in_file(NULL, old_name, new_name);
  }
  
 +int git_config_copy_section_in_file(const char *config_filename,
 +				      const char *old_name, const char *new_name)
 +{
 +	return git_config_copy_or_rename_section_in_file(config_filename,
 +					 old_name, new_name, 1);
 +}
 +
 +int git_config_copy_section(const char *old_name, const char *new_name)
 +{
 +	return git_config_copy_section_in_file(NULL, old_name, new_name);
 +}
 +
  /*
   * Call this to report error for your variable that should not
   * get a boolean value (i.e. "[my] var" means "true").
diff --combined diff.c
index d76bb937c1,85e714f6c6..6fd288420b
--- a/diff.c
+++ b/diff.c
@@@ -16,12 -16,10 +16,12 @@@
  #include "userdiff.h"
  #include "submodule-config.h"
  #include "submodule.h"
 +#include "hashmap.h"
  #include "ll-merge.h"
  #include "string-list.h"
  #include "argv-array.h"
  #include "graph.h"
 +#include "packfile.h"
  
  #ifdef NO_FAST_WORKING_DIRECTORY
  #define FAST_WORKING_DIRECTORY 0
@@@ -34,7 -32,6 +34,7 @@@ static int diff_indent_heuristic = 1
  static int diff_rename_limit_default = 400;
  static int diff_suppress_blank_empty;
  static int diff_use_color_default = -1;
 +static int diff_color_moved_default;
  static int diff_context_default = 3;
  static int diff_interhunk_context_default;
  static const char *diff_word_regex_cfg;
@@@ -59,14 -56,6 +59,14 @@@ static char diff_colors[][COLOR_MAXLEN
  	GIT_COLOR_YELLOW,	/* COMMIT */
  	GIT_COLOR_BG_RED,	/* WHITESPACE */
  	GIT_COLOR_NORMAL,	/* FUNCINFO */
 +	GIT_COLOR_BOLD_MAGENTA,	/* OLD_MOVED */
 +	GIT_COLOR_BOLD_BLUE,	/* OLD_MOVED ALTERNATIVE */
 +	GIT_COLOR_FAINT,	/* OLD_MOVED_DIM */
 +	GIT_COLOR_FAINT_ITALIC,	/* OLD_MOVED_ALTERNATIVE_DIM */
 +	GIT_COLOR_BOLD_CYAN,	/* NEW_MOVED */
 +	GIT_COLOR_BOLD_YELLOW,	/* NEW_MOVED ALTERNATIVE */
 +	GIT_COLOR_FAINT,	/* NEW_MOVED_DIM */
 +	GIT_COLOR_FAINT_ITALIC,	/* NEW_MOVED_ALTERNATIVE_DIM */
  };
  
  static NORETURN void die_want_option(const char *option_name)
@@@ -92,22 -81,6 +92,22 @@@ static int parse_diff_color_slot(const 
  		return DIFF_WHITESPACE;
  	if (!strcasecmp(var, "func"))
  		return DIFF_FUNCINFO;
 +	if (!strcasecmp(var, "oldmoved"))
 +		return DIFF_FILE_OLD_MOVED;
 +	if (!strcasecmp(var, "oldmovedalternative"))
 +		return DIFF_FILE_OLD_MOVED_ALT;
 +	if (!strcasecmp(var, "oldmoveddimmed"))
 +		return DIFF_FILE_OLD_MOVED_DIM;
 +	if (!strcasecmp(var, "oldmovedalternativedimmed"))
 +		return DIFF_FILE_OLD_MOVED_ALT_DIM;
 +	if (!strcasecmp(var, "newmoved"))
 +		return DIFF_FILE_NEW_MOVED;
 +	if (!strcasecmp(var, "newmovedalternative"))
 +		return DIFF_FILE_NEW_MOVED_ALT;
 +	if (!strcasecmp(var, "newmoveddimmed"))
 +		return DIFF_FILE_NEW_MOVED_DIM;
 +	if (!strcasecmp(var, "newmovedalternativedimmed"))
 +		return DIFF_FILE_NEW_MOVED_ALT_DIM;
  	return -1;
  }
  
@@@ -256,44 -229,12 +256,44 @@@ int git_diff_heuristic_config(const cha
  	return 0;
  }
  
 +static int parse_color_moved(const char *arg)
 +{
 +	switch (git_parse_maybe_bool(arg)) {
 +	case 0:
 +		return COLOR_MOVED_NO;
 +	case 1:
 +		return COLOR_MOVED_DEFAULT;
 +	default:
 +		break;
 +	}
 +
 +	if (!strcmp(arg, "no"))
 +		return COLOR_MOVED_NO;
 +	else if (!strcmp(arg, "plain"))
 +		return COLOR_MOVED_PLAIN;
 +	else if (!strcmp(arg, "zebra"))
 +		return COLOR_MOVED_ZEBRA;
 +	else if (!strcmp(arg, "default"))
 +		return COLOR_MOVED_DEFAULT;
 +	else if (!strcmp(arg, "dimmed_zebra"))
 +		return COLOR_MOVED_ZEBRA_DIM;
 +	else
 +		return error(_("color moved setting must be one of 'no', 'default', 'zebra', 'dimmed_zebra', 'plain'"));
 +}
 +
  int git_diff_ui_config(const char *var, const char *value, void *cb)
  {
  	if (!strcmp(var, "diff.color") || !strcmp(var, "color.diff")) {
  		diff_use_color_default = git_config_colorbool(var, value);
  		return 0;
  	}
 +	if (!strcmp(var, "diff.colormoved")) {
 +		int cm = parse_color_moved(value);
 +		if (cm < 0)
 +			return -1;
 +		diff_color_moved_default = cm;
 +		return 0;
 +	}
  	if (!strcmp(var, "diff.context")) {
  		diff_context_default = git_config_int(var, value);
  		if (diff_context_default < 0)
@@@ -358,6 -299,9 +358,9 @@@
  		return 0;
  	}
  
+ 	if (git_color_config(var, value, cb) < 0)
+ 		return -1;
+ 
  	return git_diff_basic_config(var, value, cb);
  }
  
@@@ -402,6 -346,9 +405,6 @@@ int git_diff_basic_config(const char *v
  		return 0;
  	}
  
 -	if (starts_with(var, "submodule."))
 -		return parse_submodule_config_option(var, value);
 -
  	if (git_diff_heuristic_config(var, value, cb) < 0)
  		return -1;
  
@@@ -459,9 -406,11 +462,9 @@@ static struct diff_tempfile 
  	 * If this diff_tempfile instance refers to a temporary file,
  	 * this tempfile object is used to manage its lifetime.
  	 */
 -	struct tempfile tempfile;
 +	struct tempfile *tempfile;
  } diff_temp[2];
  
 -typedef unsigned long (*sane_truncate_fn)(char *line, unsigned long len);
 -
  struct emit_callback {
  	int color_diff;
  	unsigned ws_rule;
@@@ -469,6 -418,7 +472,6 @@@
  	int blank_at_eof_in_postimage;
  	int lno_in_preimage;
  	int lno_in_postimage;
 -	sane_truncate_fn truncate;
  	const char **label_path;
  	struct diff_words_data *diff_words;
  	struct diff_options *opt;
@@@ -610,737 -560,68 +613,737 @@@ static void emit_line(struct diff_optio
  	emit_line_0(o, set, reset, line[0], line+1, len-1);
  }
  
 -static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
 +enum diff_symbol {
 +	DIFF_SYMBOL_BINARY_DIFF_HEADER,
 +	DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,
 +	DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,
 +	DIFF_SYMBOL_BINARY_DIFF_BODY,
 +	DIFF_SYMBOL_BINARY_DIFF_FOOTER,
 +	DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,
 +	DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
 +	DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
 +	DIFF_SYMBOL_STATS_LINE,
 +	DIFF_SYMBOL_WORD_DIFF,
 +	DIFF_SYMBOL_STAT_SEP,
 +	DIFF_SYMBOL_SUMMARY,
 +	DIFF_SYMBOL_SUBMODULE_ADD,
 +	DIFF_SYMBOL_SUBMODULE_DEL,
 +	DIFF_SYMBOL_SUBMODULE_UNTRACKED,
 +	DIFF_SYMBOL_SUBMODULE_MODIFIED,
 +	DIFF_SYMBOL_SUBMODULE_HEADER,
 +	DIFF_SYMBOL_SUBMODULE_ERROR,
 +	DIFF_SYMBOL_SUBMODULE_PIPETHROUGH,
 +	DIFF_SYMBOL_REWRITE_DIFF,
 +	DIFF_SYMBOL_BINARY_FILES,
 +	DIFF_SYMBOL_HEADER,
 +	DIFF_SYMBOL_FILEPAIR_PLUS,
 +	DIFF_SYMBOL_FILEPAIR_MINUS,
 +	DIFF_SYMBOL_WORDS_PORCELAIN,
 +	DIFF_SYMBOL_WORDS,
 +	DIFF_SYMBOL_CONTEXT,
 +	DIFF_SYMBOL_CONTEXT_INCOMPLETE,
 +	DIFF_SYMBOL_PLUS,
 +	DIFF_SYMBOL_MINUS,
 +	DIFF_SYMBOL_NO_LF_EOF,
 +	DIFF_SYMBOL_CONTEXT_FRAGINFO,
 +	DIFF_SYMBOL_CONTEXT_MARKER,
 +	DIFF_SYMBOL_SEPARATOR
 +};
 +/*
 + * Flags for content lines:
 + * 0..12 are whitespace rules
 + * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT
 + * 16 is marking if the line is blank at EOF
 + */
 +#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF	(1<<16)
 +#define DIFF_SYMBOL_MOVED_LINE			(1<<17)
 +#define DIFF_SYMBOL_MOVED_LINE_ALT		(1<<18)
 +#define DIFF_SYMBOL_MOVED_LINE_UNINTERESTING	(1<<19)
 +#define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)
 +
 +/*
 + * This struct is used when we need to buffer the output of the diff output.
 + *
 + * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer
 + * into the pre/post image file. This pointer could be a union with the
 + * line pointer. By storing an offset into the file instead of the literal line,
 + * we can decrease the memory footprint for the buffered output. At first we
 + * may want to only have indirection for the content lines, but we could also
 + * enhance the state for emitting prefabricated lines, e.g. the similarity
 + * score line or hunk/file headers would only need to store a number or path
 + * and then the output can be constructed later on depending on state.
 + */
 +struct emitted_diff_symbol {
 +	const char *line;
 +	int len;
 +	int flags;
 +	enum diff_symbol s;
 +};
 +#define EMITTED_DIFF_SYMBOL_INIT {NULL}
 +
 +struct emitted_diff_symbols {
 +	struct emitted_diff_symbol *buf;
 +	int nr, alloc;
 +};
 +#define EMITTED_DIFF_SYMBOLS_INIT {NULL, 0, 0}
 +
 +static void append_emitted_diff_symbol(struct diff_options *o,
 +				       struct emitted_diff_symbol *e)
  {
 -	if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
 -	      ecbdata->blank_at_eof_in_preimage &&
 -	      ecbdata->blank_at_eof_in_postimage &&
 -	      ecbdata->blank_at_eof_in_preimage <= ecbdata->lno_in_preimage &&
 -	      ecbdata->blank_at_eof_in_postimage <= ecbdata->lno_in_postimage))
 -		return 0;
 -	return ws_blank_line(line, len, ecbdata->ws_rule);
 +	struct emitted_diff_symbol *f;
 +
 +	ALLOC_GROW(o->emitted_symbols->buf,
 +		   o->emitted_symbols->nr + 1,
 +		   o->emitted_symbols->alloc);
 +	f = &o->emitted_symbols->buf[o->emitted_symbols->nr++];
 +
 +	memcpy(f, e, sizeof(struct emitted_diff_symbol));
 +	f->line = e->line ? xmemdupz(e->line, e->len) : NULL;
  }
  
 -static void emit_line_checked(const char *reset,
 -			      struct emit_callback *ecbdata,
 -			      const char *line, int len,
 -			      enum color_diff color,
 -			      unsigned ws_error_highlight,
 -			      char sign)
 +struct moved_entry {
 +	struct hashmap_entry ent;
 +	const struct emitted_diff_symbol *es;
 +	struct moved_entry *next_line;
 +};
 +
 +static int next_byte(const char **cp, const char **endp,
 +		     const struct diff_options *diffopt)
 +{
 +	int retval;
 +
 +	if (*cp > *endp)
 +		return -1;
 +
 +	if (isspace(**cp)) {
 +		if (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_CHANGE)) {
 +			while (*cp < *endp && isspace(**cp))
 +				(*cp)++;
 +			/*
 +			 * After skipping a couple of whitespaces,
 +			 * we still have to account for one space.
 +			 */
 +			return (int)' ';
 +		}
 +
 +		if (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE)) {
 +			while (*cp < *endp && isspace(**cp))
 +				(*cp)++;
 +			/* return the first non-ws character via the usual below */
 +		}
 +	}
 +
 +	retval = (unsigned char)(**cp);
 +	(*cp)++;
 +	return retval;
 +}
 +
 +static int moved_entry_cmp(const struct diff_options *diffopt,
 +			   const struct moved_entry *a,
 +			   const struct moved_entry *b,
 +			   const void *keydata)
 +{
 +	const char *ap = a->es->line, *ae = a->es->line + a->es->len;
 +	const char *bp = b->es->line, *be = b->es->line + b->es->len;
 +
 +	if (!(diffopt->xdl_opts & XDF_WHITESPACE_FLAGS))
 +		return a->es->len != b->es->len  || memcmp(ap, bp, a->es->len);
 +
 +	if (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_AT_EOL)) {
 +		while (ae > ap && isspace(*ae))
 +			ae--;
 +		while (be > bp && isspace(*be))
 +			be--;
 +	}
 +
 +	while (1) {
 +		int ca, cb;
 +		ca = next_byte(&ap, &ae, diffopt);
 +		cb = next_byte(&bp, &be, diffopt);
 +		if (ca != cb)
 +			return 1;
 +		if (ca < 0)
 +			return 0;
 +	}
 +}
 +
 +static unsigned get_string_hash(struct emitted_diff_symbol *es, struct diff_options *o)
 +{
 +	if (o->xdl_opts & XDF_WHITESPACE_FLAGS) {
 +		static struct strbuf sb = STRBUF_INIT;
 +		const char *ap = es->line, *ae = es->line + es->len;
 +		int c;
 +
 +		strbuf_reset(&sb);
 +		while (ae > ap && isspace(*ae))
 +			ae--;
 +		while ((c = next_byte(&ap, &ae, o)) > 0)
 +			strbuf_addch(&sb, c);
 +
 +		return memhash(sb.buf, sb.len);
 +	} else {
 +		return memhash(es->line, es->len);
 +	}
 +}
 +
 +static struct moved_entry *prepare_entry(struct diff_options *o,
 +					 int line_no)
 +{
 +	struct moved_entry *ret = xmalloc(sizeof(*ret));
 +	struct emitted_diff_symbol *l = &o->emitted_symbols->buf[line_no];
 +
 +	ret->ent.hash = get_string_hash(l, o);
 +	ret->es = l;
 +	ret->next_line = NULL;
 +
 +	return ret;
 +}
 +
 +static void add_lines_to_move_detection(struct diff_options *o,
 +					struct hashmap *add_lines,
 +					struct hashmap *del_lines)
 +{
 +	struct moved_entry *prev_line = NULL;
 +
 +	int n;
 +	for (n = 0; n < o->emitted_symbols->nr; n++) {
 +		struct hashmap *hm;
 +		struct moved_entry *key;
 +
 +		switch (o->emitted_symbols->buf[n].s) {
 +		case DIFF_SYMBOL_PLUS:
 +			hm = add_lines;
 +			break;
 +		case DIFF_SYMBOL_MINUS:
 +			hm = del_lines;
 +			break;
 +		default:
 +			prev_line = NULL;
 +			continue;
 +		}
 +
 +		key = prepare_entry(o, n);
 +		if (prev_line && prev_line->es->s == o->emitted_symbols->buf[n].s)
 +			prev_line->next_line = key;
 +
 +		hashmap_add(hm, key);
 +		prev_line = key;
 +	}
 +}
 +
 +static int shrink_potential_moved_blocks(struct moved_entry **pmb,
 +					 int pmb_nr)
 +{
 +	int lp, rp;
 +
 +	/* Shrink the set of potential block to the remaining running */
 +	for (lp = 0, rp = pmb_nr - 1; lp <= rp;) {
 +		while (lp < pmb_nr && pmb[lp])
 +			lp++;
 +		/* lp points at the first NULL now */
 +
 +		while (rp > -1 && !pmb[rp])
 +			rp--;
 +		/* rp points at the last non-NULL */
 +
 +		if (lp < pmb_nr && rp > -1 && lp < rp) {
 +			pmb[lp] = pmb[rp];
 +			pmb[rp] = NULL;
 +			rp--;
 +			lp++;
 +		}
 +	}
 +
 +	/* Remember the number of running sets */
 +	return rp + 1;
 +}
 +
 +/*
 + * If o->color_moved is COLOR_MOVED_PLAIN, this function does nothing.
 + *
 + * Otherwise, if the last block has fewer alphanumeric characters than
 + * COLOR_MOVED_MIN_ALNUM_COUNT, unset DIFF_SYMBOL_MOVED_LINE on all lines in
 + * that block.
 + *
 + * The last block consists of the (n - block_length)'th line up to but not
 + * including the nth line.
 + *
 + * NEEDSWORK: This uses the same heuristic as blame_entry_score() in blame.c.
 + * Think of a way to unify them.
 + */
 +static void adjust_last_block(struct diff_options *o, int n, int block_length)
 +{
 +	int i, alnum_count = 0;
 +	if (o->color_moved == COLOR_MOVED_PLAIN)
 +		return;
 +	for (i = 1; i < block_length + 1; i++) {
 +		const char *c = o->emitted_symbols->buf[n - i].line;
 +		for (; *c; c++) {
 +			if (!isalnum(*c))
 +				continue;
 +			alnum_count++;
 +			if (alnum_count >= COLOR_MOVED_MIN_ALNUM_COUNT)
 +				return;
 +		}
 +	}
 +	for (i = 1; i < block_length + 1; i++)
 +		o->emitted_symbols->buf[n - i].flags &= ~DIFF_SYMBOL_MOVED_LINE;
 +}
 +
 +/* Find blocks of moved code, delegate actual coloring decision to helper */
 +static void mark_color_as_moved(struct diff_options *o,
 +				struct hashmap *add_lines,
 +				struct hashmap *del_lines)
 +{
 +	struct moved_entry **pmb = NULL; /* potentially moved blocks */
 +	int pmb_nr = 0, pmb_alloc = 0;
 +	int n, flipped_block = 1, block_length = 0;
 +
 +
 +	for (n = 0; n < o->emitted_symbols->nr; n++) {
 +		struct hashmap *hm = NULL;
 +		struct moved_entry *key;
 +		struct moved_entry *match = NULL;
 +		struct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];
 +		int i;
 +
 +		switch (l->s) {
 +		case DIFF_SYMBOL_PLUS:
 +			hm = del_lines;
 +			key = prepare_entry(o, n);
 +			match = hashmap_get(hm, key, o);
 +			free(key);
 +			break;
 +		case DIFF_SYMBOL_MINUS:
 +			hm = add_lines;
 +			key = prepare_entry(o, n);
 +			match = hashmap_get(hm, key, o);
 +			free(key);
 +			break;
 +		default:
 +			flipped_block = 1;
 +		}
 +
 +		if (!match) {
 +			adjust_last_block(o, n, block_length);
 +			pmb_nr = 0;
 +			block_length = 0;
 +			continue;
 +		}
 +
 +		l->flags |= DIFF_SYMBOL_MOVED_LINE;
 +
 +		if (o->color_moved == COLOR_MOVED_PLAIN)
 +			continue;
 +
 +		/* Check any potential block runs, advance each or nullify */
 +		for (i = 0; i < pmb_nr; i++) {
 +			struct moved_entry *p = pmb[i];
 +			struct moved_entry *pnext = (p && p->next_line) ?
 +					p->next_line : NULL;
 +			if (pnext && !hm->cmpfn(o, pnext, match, NULL)) {
 +				pmb[i] = p->next_line;
 +			} else {
 +				pmb[i] = NULL;
 +			}
 +		}
 +
 +		pmb_nr = shrink_potential_moved_blocks(pmb, pmb_nr);
 +
 +		if (pmb_nr == 0) {
 +			/*
 +			 * The current line is the start of a new block.
 +			 * Setup the set of potential blocks.
 +			 */
 +			for (; match; match = hashmap_get_next(hm, match)) {
 +				ALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);
 +				pmb[pmb_nr++] = match;
 +			}
 +
 +			flipped_block = (flipped_block + 1) % 2;
 +
 +			adjust_last_block(o, n, block_length);
 +			block_length = 0;
 +		}
 +
 +		block_length++;
 +
 +		if (flipped_block)
 +			l->flags |= DIFF_SYMBOL_MOVED_LINE_ALT;
 +	}
 +	adjust_last_block(o, n, block_length);
 +
 +	free(pmb);
 +}
 +
 +#define DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK \
 +  (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ALT)
 +static void dim_moved_lines(struct diff_options *o)
 +{
 +	int n;
 +	for (n = 0; n < o->emitted_symbols->nr; n++) {
 +		struct emitted_diff_symbol *prev = (n != 0) ?
 +				&o->emitted_symbols->buf[n - 1] : NULL;
 +		struct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];
 +		struct emitted_diff_symbol *next =
 +				(n < o->emitted_symbols->nr - 1) ?
 +				&o->emitted_symbols->buf[n + 1] : NULL;
 +
 +		/* Not a plus or minus line? */
 +		if (l->s != DIFF_SYMBOL_PLUS && l->s != DIFF_SYMBOL_MINUS)
 +			continue;
 +
 +		/* Not a moved line? */
 +		if (!(l->flags & DIFF_SYMBOL_MOVED_LINE))
 +			continue;
 +
 +		/*
 +		 * If prev or next are not a plus or minus line,
 +		 * pretend they don't exist
 +		 */
 +		if (prev && prev->s != DIFF_SYMBOL_PLUS &&
 +			    prev->s != DIFF_SYMBOL_MINUS)
 +			prev = NULL;
 +		if (next && next->s != DIFF_SYMBOL_PLUS &&
 +			    next->s != DIFF_SYMBOL_MINUS)
 +			next = NULL;
 +
 +		/* Inside a block? */
 +		if ((prev &&
 +		    (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==
 +		    (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK)) &&
 +		    (next &&
 +		    (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==
 +		    (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK))) {
 +			l->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;
 +			continue;
 +		}
 +
 +		/* Check if we are at an interesting bound: */
 +		if (prev && (prev->flags & DIFF_SYMBOL_MOVED_LINE) &&
 +		    (prev->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=
 +		       (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))
 +			continue;
 +		if (next && (next->flags & DIFF_SYMBOL_MOVED_LINE) &&
 +		    (next->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=
 +		       (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))
 +			continue;
 +
 +		/*
 +		 * The boundary to prev and next are not interesting,
 +		 * so this line is not interesting as a whole
 +		 */
 +		l->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;
 +	}
 +}
 +
 +static void emit_line_ws_markup(struct diff_options *o,
 +				const char *set, const char *reset,
 +				const char *line, int len, char sign,
 +				unsigned ws_rule, int blank_at_eof)
  {
 -	const char *set = diff_get_color(ecbdata->color_diff, color);
  	const char *ws = NULL;
  
 -	if (ecbdata->opt->ws_error_highlight & ws_error_highlight) {
 -		ws = diff_get_color(ecbdata->color_diff, DIFF_WHITESPACE);
 +	if (o->ws_error_highlight & ws_rule) {
 +		ws = diff_get_color_opt(o, DIFF_WHITESPACE);
  		if (!*ws)
  			ws = NULL;
  	}
  
  	if (!ws)
 -		emit_line_0(ecbdata->opt, set, reset, sign, line, len);
 -	else if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))
 +		emit_line_0(o, set, reset, sign, line, len);
 +	else if (blank_at_eof)
  		/* Blank line at EOF - paint '+' as well */
 -		emit_line_0(ecbdata->opt, ws, reset, sign, line, len);
 +		emit_line_0(o, ws, reset, sign, line, len);
  	else {
  		/* Emit just the prefix, then the rest. */
 -		emit_line_0(ecbdata->opt, set, reset, sign, "", 0);
 -		ws_check_emit(line, len, ecbdata->ws_rule,
 -			      ecbdata->opt->file, set, reset, ws);
 +		emit_line_0(o, set, reset, sign, "", 0);
 +		ws_check_emit(line, len, ws_rule,
 +			      o->file, set, reset, ws);
 +	}
 +}
 +
 +static void emit_diff_symbol_from_struct(struct diff_options *o,
 +					 struct emitted_diff_symbol *eds)
 +{
 +	static const char *nneof = " No newline at end of file\n";
 +	const char *context, *reset, *set, *meta, *fraginfo;
 +	struct strbuf sb = STRBUF_INIT;
 +
 +	enum diff_symbol s = eds->s;
 +	const char *line = eds->line;
 +	int len = eds->len;
 +	unsigned flags = eds->flags;
 +
 +	switch (s) {
 +	case DIFF_SYMBOL_NO_LF_EOF:
 +		context = diff_get_color_opt(o, DIFF_CONTEXT);
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		putc('\n', o->file);
 +		emit_line_0(o, context, reset, '\\',
 +			    nneof, strlen(nneof));
 +		break;
 +	case DIFF_SYMBOL_SUBMODULE_HEADER:
 +	case DIFF_SYMBOL_SUBMODULE_ERROR:
 +	case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
 +	case DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:
 +	case DIFF_SYMBOL_SUMMARY:
 +	case DIFF_SYMBOL_STATS_LINE:
 +	case DIFF_SYMBOL_BINARY_DIFF_BODY:
 +	case DIFF_SYMBOL_CONTEXT_FRAGINFO:
 +		emit_line(o, "", "", line, len);
 +		break;
 +	case DIFF_SYMBOL_CONTEXT_INCOMPLETE:
 +	case DIFF_SYMBOL_CONTEXT_MARKER:
 +		context = diff_get_color_opt(o, DIFF_CONTEXT);
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		emit_line(o, context, reset, line, len);
 +		break;
 +	case DIFF_SYMBOL_SEPARATOR:
 +		fprintf(o->file, "%s%c",
 +			diff_line_prefix(o),
 +			o->line_termination);
 +		break;
 +	case DIFF_SYMBOL_CONTEXT:
 +		set = diff_get_color_opt(o, DIFF_CONTEXT);
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		emit_line_ws_markup(o, set, reset, line, len, ' ',
 +				    flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);
 +		break;
 +	case DIFF_SYMBOL_PLUS:
 +		switch (flags & (DIFF_SYMBOL_MOVED_LINE |
 +				 DIFF_SYMBOL_MOVED_LINE_ALT |
 +				 DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {
 +		case DIFF_SYMBOL_MOVED_LINE |
 +		     DIFF_SYMBOL_MOVED_LINE_ALT |
 +		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
 +			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT_DIM);
 +			break;
 +		case DIFF_SYMBOL_MOVED_LINE |
 +		     DIFF_SYMBOL_MOVED_LINE_ALT:
 +			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);
 +			break;
 +		case DIFF_SYMBOL_MOVED_LINE |
 +		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
 +			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_DIM);
 +			break;
 +		case DIFF_SYMBOL_MOVED_LINE:
 +			set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);
 +			break;
 +		default:
 +			set = diff_get_color_opt(o, DIFF_FILE_NEW);
 +		}
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		emit_line_ws_markup(o, set, reset, line, len, '+',
 +				    flags & DIFF_SYMBOL_CONTENT_WS_MASK,
 +				    flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
 +		break;
 +	case DIFF_SYMBOL_MINUS:
 +		switch (flags & (DIFF_SYMBOL_MOVED_LINE |
 +				 DIFF_SYMBOL_MOVED_LINE_ALT |
 +				 DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {
 +		case DIFF_SYMBOL_MOVED_LINE |
 +		     DIFF_SYMBOL_MOVED_LINE_ALT |
 +		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
 +			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT_DIM);
 +			break;
 +		case DIFF_SYMBOL_MOVED_LINE |
 +		     DIFF_SYMBOL_MOVED_LINE_ALT:
 +			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);
 +			break;
 +		case DIFF_SYMBOL_MOVED_LINE |
 +		     DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
 +			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_DIM);
 +			break;
 +		case DIFF_SYMBOL_MOVED_LINE:
 +			set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);
 +			break;
 +		default:
 +			set = diff_get_color_opt(o, DIFF_FILE_OLD);
 +		}
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		emit_line_ws_markup(o, set, reset, line, len, '-',
 +				    flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
 +		break;
 +	case DIFF_SYMBOL_WORDS_PORCELAIN:
 +		context = diff_get_color_opt(o, DIFF_CONTEXT);
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		emit_line(o, context, reset, line, len);
 +		fputs("~\n", o->file);
 +		break;
 +	case DIFF_SYMBOL_WORDS:
 +		context = diff_get_color_opt(o, DIFF_CONTEXT);
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		/*
 +		 * Skip the prefix character, if any.  With
 +		 * diff_suppress_blank_empty, there may be
 +		 * none.
 +		 */
 +		if (line[0] != '\n') {
 +			line++;
 +			len--;
 +		}
 +		emit_line(o, context, reset, line, len);
 +		break;
 +	case DIFF_SYMBOL_FILEPAIR_PLUS:
 +		meta = diff_get_color_opt(o, DIFF_METAINFO);
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		fprintf(o->file, "%s%s+++ %s%s%s\n", diff_line_prefix(o), meta,
 +			line, reset,
 +			strchr(line, ' ') ? "\t" : "");
 +		break;
 +	case DIFF_SYMBOL_FILEPAIR_MINUS:
 +		meta = diff_get_color_opt(o, DIFF_METAINFO);
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		fprintf(o->file, "%s%s--- %s%s%s\n", diff_line_prefix(o), meta,
 +			line, reset,
 +			strchr(line, ' ') ? "\t" : "");
 +		break;
 +	case DIFF_SYMBOL_BINARY_FILES:
 +	case DIFF_SYMBOL_HEADER:
 +		fprintf(o->file, "%s", line);
 +		break;
 +	case DIFF_SYMBOL_BINARY_DIFF_HEADER:
 +		fprintf(o->file, "%sGIT binary patch\n", diff_line_prefix(o));
 +		break;
 +	case DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA:
 +		fprintf(o->file, "%sdelta %s\n", diff_line_prefix(o), line);
 +		break;
 +	case DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL:
 +		fprintf(o->file, "%sliteral %s\n", diff_line_prefix(o), line);
 +		break;
 +	case DIFF_SYMBOL_BINARY_DIFF_FOOTER:
 +		fputs(diff_line_prefix(o), o->file);
 +		fputc('\n', o->file);
 +		break;
 +	case DIFF_SYMBOL_REWRITE_DIFF:
 +		fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		emit_line(o, fraginfo, reset, line, len);
 +		break;
 +	case DIFF_SYMBOL_SUBMODULE_ADD:
 +		set = diff_get_color_opt(o, DIFF_FILE_NEW);
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		emit_line(o, set, reset, line, len);
 +		break;
 +	case DIFF_SYMBOL_SUBMODULE_DEL:
 +		set = diff_get_color_opt(o, DIFF_FILE_OLD);
 +		reset = diff_get_color_opt(o, DIFF_RESET);
 +		emit_line(o, set, reset, line, len);
 +		break;
 +	case DIFF_SYMBOL_SUBMODULE_UNTRACKED:
 +		fprintf(o->file, "%sSubmodule %s contains untracked content\n",
 +			diff_line_prefix(o), line);
 +		break;
 +	case DIFF_SYMBOL_SUBMODULE_MODIFIED:
 +		fprintf(o->file, "%sSubmodule %s contains modified content\n",
 +			diff_line_prefix(o), line);
 +		break;
 +	case DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:
 +		emit_line(o, "", "", " 0 files changed\n",
 +			  strlen(" 0 files changed\n"));
 +		break;
 +	case DIFF_SYMBOL_STATS_SUMMARY_ABBREV:
 +		emit_line(o, "", "", " ...\n", strlen(" ...\n"));
 +		break;
 +	case DIFF_SYMBOL_WORD_DIFF:
 +		fprintf(o->file, "%.*s", len, line);
 +		break;
 +	case DIFF_SYMBOL_STAT_SEP:
 +		fputs(o->stat_sep, o->file);
 +		break;
 +	default:
 +		die("BUG: unknown diff symbol");
  	}
 +	strbuf_release(&sb);
 +}
 +
 +static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 +			     const char *line, int len, unsigned flags)
 +{
 +	struct emitted_diff_symbol e = {line, len, flags, s};
 +
 +	if (o->emitted_symbols)
 +		append_emitted_diff_symbol(o, &e);
 +	else
 +		emit_diff_symbol_from_struct(o, &e);
 +}
 +
 +void diff_emit_submodule_del(struct diff_options *o, const char *line)
 +{
 +	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);
 +}
 +
 +void diff_emit_submodule_add(struct diff_options *o, const char *line)
 +{
 +	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ADD, line, strlen(line), 0);
 +}
 +
 +void diff_emit_submodule_untracked(struct diff_options *o, const char *path)
 +{
 +	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_UNTRACKED,
 +			 path, strlen(path), 0);
 +}
 +
 +void diff_emit_submodule_modified(struct diff_options *o, const char *path)
 +{
 +	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_MODIFIED,
 +			 path, strlen(path), 0);
 +}
 +
 +void diff_emit_submodule_header(struct diff_options *o, const char *header)
 +{
 +	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_HEADER,
 +			 header, strlen(header), 0);
 +}
 +
 +void diff_emit_submodule_error(struct diff_options *o, const char *err)
 +{
 +	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ERROR, err, strlen(err), 0);
 +}
 +
 +void diff_emit_submodule_pipethrough(struct diff_options *o,
 +				     const char *line, int len)
 +{
 +	emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_PIPETHROUGH, line, len, 0);
 +}
 +
 +static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
 +{
 +	if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
 +	      ecbdata->blank_at_eof_in_preimage &&
 +	      ecbdata->blank_at_eof_in_postimage &&
 +	      ecbdata->blank_at_eof_in_preimage <= ecbdata->lno_in_preimage &&
 +	      ecbdata->blank_at_eof_in_postimage <= ecbdata->lno_in_postimage))
 +		return 0;
 +	return ws_blank_line(line, len, ecbdata->ws_rule);
  }
  
  static void emit_add_line(const char *reset,
  			  struct emit_callback *ecbdata,
  			  const char *line, int len)
  {
 -	emit_line_checked(reset, ecbdata, line, len,
 -			  DIFF_FILE_NEW, WSEH_NEW, '+');
 +	unsigned flags = WSEH_NEW | ecbdata->ws_rule;
 +	if (new_blank_line_at_eof(ecbdata, line, len))
 +		flags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;
 +
 +	emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);
  }
  
  static void emit_del_line(const char *reset,
  			  struct emit_callback *ecbdata,
  			  const char *line, int len)
  {
 -	emit_line_checked(reset, ecbdata, line, len,
 -			  DIFF_FILE_OLD, WSEH_OLD, '-');
 +	unsigned flags = WSEH_OLD | ecbdata->ws_rule;
 +	emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_MINUS, line, len, flags);
  }
  
  static void emit_context_line(const char *reset,
  			      struct emit_callback *ecbdata,
  			      const char *line, int len)
  {
 -	emit_line_checked(reset, ecbdata, line, len,
 -			  DIFF_CONTEXT, WSEH_CONTEXT, ' ');
 +	unsigned flags = WSEH_CONTEXT | ecbdata->ws_rule;
 +	emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_CONTEXT, line, len, flags);
  }
  
  static void emit_hunk_header(struct emit_callback *ecbdata,
@@@ -1363,8 -644,7 +1366,8 @@@
  	if (len < 10 ||
  	    memcmp(line, atat, 2) ||
  	    !(ep = memmem(line + 2, len - 2, atat, 2))) {
 -		emit_line(ecbdata->opt, context, reset, line, len);
 +		emit_diff_symbol(ecbdata->opt,
 +				 DIFF_SYMBOL_CONTEXT_MARKER, line, len, 0);
  		return;
  	}
  	ep += 2; /* skip over @@ */
@@@ -1398,9 -678,7 +1401,9 @@@
  	}
  
  	strbuf_add(&msgbuf, line + len, org_len - len);
 -	emit_line(ecbdata->opt, "", "", msgbuf.buf, msgbuf.len);
 +	strbuf_complete_line(&msgbuf);
 +	emit_diff_symbol(ecbdata->opt,
 +			 DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len, 0);
  	strbuf_release(&msgbuf);
  }
  
@@@ -1416,23 -694,23 +1419,23 @@@ static void remove_tempfile(void
  {
  	int i;
  	for (i = 0; i < ARRAY_SIZE(diff_temp); i++) {
 -		if (is_tempfile_active(&diff_temp[i].tempfile))
 +		if (is_tempfile_active(diff_temp[i].tempfile))
  			delete_tempfile(&diff_temp[i].tempfile);
  		diff_temp[i].name = NULL;
  	}
  }
  
 -static void print_line_count(FILE *file, int count)
 +static void add_line_count(struct strbuf *out, int count)
  {
  	switch (count) {
  	case 0:
 -		fprintf(file, "0,0");
 +		strbuf_addstr(out, "0,0");
  		break;
  	case 1:
 -		fprintf(file, "1");
 +		strbuf_addstr(out, "1");
  		break;
  	default:
 -		fprintf(file, "1,%d", count);
 +		strbuf_addf(out, "1,%d", count);
  		break;
  	}
  }
@@@ -1441,6 -719,7 +1444,6 @@@ static void emit_rewrite_lines(struct e
  			       int prefix, const char *data, int size)
  {
  	const char *endp = NULL;
 -	static const char *nneof = " No newline at end of file\n";
  	const char *reset = diff_get_color(ecb->color_diff, DIFF_RESET);
  
  	while (0 < size) {
@@@ -1458,8 -737,13 +1461,8 @@@
  		size -= len;
  		data += len;
  	}
 -	if (!endp) {
 -		const char *context = diff_get_color(ecb->color_diff,
 -						     DIFF_CONTEXT);
 -		putc('\n', ecb->opt->file);
 -		emit_line_0(ecb->opt, context, reset, '\\',
 -			    nneof, strlen(nneof));
 -	}
 +	if (!endp)
 +		emit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0, 0);
  }
  
  static void emit_rewrite_diff(const char *name_a,
@@@ -1471,12 -755,16 +1474,12 @@@
  			      struct diff_options *o)
  {
  	int lc_a, lc_b;
 -	const char *name_a_tab, *name_b_tab;
 -	const char *metainfo = diff_get_color(o->use_color, DIFF_METAINFO);
 -	const char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
 -	const char *reset = diff_get_color(o->use_color, DIFF_RESET);
  	static struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;
  	const char *a_prefix, *b_prefix;
  	char *data_one, *data_two;
  	size_t size_one, size_two;
  	struct emit_callback ecbdata;
 -	const char *line_prefix = diff_line_prefix(o);
 +	struct strbuf out = STRBUF_INIT;
  
  	if (diff_mnemonic_prefix && DIFF_OPT_TST(o, REVERSE_DIFF)) {
  		a_prefix = o->b_prefix;
@@@ -1488,6 -776,8 +1491,6 @@@
  
  	name_a += (*name_a == '/');
  	name_b += (*name_b == '/');
 -	name_a_tab = strchr(name_a, ' ') ? "\t" : "";
 -	name_b_tab = strchr(name_b, ' ') ? "\t" : "";
  
  	strbuf_reset(&a_name);
  	strbuf_reset(&b_name);
@@@ -1514,23 -804,18 +1517,23 @@@
  
  	lc_a = count_lines(data_one, size_one);
  	lc_b = count_lines(data_two, size_two);
 -	fprintf(o->file,
 -		"%s%s--- %s%s%s\n%s%s+++ %s%s%s\n%s%s@@ -",
 -		line_prefix, metainfo, a_name.buf, name_a_tab, reset,
 -		line_prefix, metainfo, b_name.buf, name_b_tab, reset,
 -		line_prefix, fraginfo);
 +
 +	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,
 +			 a_name.buf, a_name.len, 0);
 +	emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,
 +			 b_name.buf, b_name.len, 0);
 +
 +	strbuf_addstr(&out, "@@ -");
  	if (!o->irreversible_delete)
 -		print_line_count(o->file, lc_a);
 +		add_line_count(&out, lc_a);
  	else
 -		fprintf(o->file, "?,?");
 -	fprintf(o->file, " +");
 -	print_line_count(o->file, lc_b);
 -	fprintf(o->file, " @@%s\n", reset);
 +		strbuf_addstr(&out, "?,?");
 +	strbuf_addstr(&out, " +");
 +	add_line_count(&out, lc_b);
 +	strbuf_addstr(&out, " @@\n");
 +	emit_diff_symbol(o, DIFF_SYMBOL_REWRITE_DIFF, out.buf, out.len, 0);
 +	strbuf_release(&out);
 +
  	if (lc_a && !o->irreversible_delete)
  		emit_rewrite_lines(&ecbdata, '-', data_one, size_one);
  	if (lc_b)
@@@ -1543,7 -828,7 +1546,7 @@@
  
  struct diff_words_buffer {
  	mmfile_t text;
 -	long alloc;
 +	unsigned long alloc;
  	struct diff_words_orig {
  		const char *begin, *end;
  	} *orig;
@@@ -1590,49 -875,37 +1593,49 @@@ struct diff_words_data 
  	struct diff_words_style *style;
  };
  
 -static int fn_out_diff_words_write_helper(FILE *fp,
 +static int fn_out_diff_words_write_helper(struct diff_options *o,
  					  struct diff_words_style_elem *st_el,
  					  const char *newline,
 -					  size_t count, const char *buf,
 -					  const char *line_prefix)
 +					  size_t count, const char *buf)
  {
  	int print = 0;
 +	struct strbuf sb = STRBUF_INIT;
  
  	while (count) {
  		char *p = memchr(buf, '\n', count);
  		if (print)
 -			fputs(line_prefix, fp);
 +			strbuf_addstr(&sb, diff_line_prefix(o));
 +
  		if (p != buf) {
 -			if (st_el->color && fputs(st_el->color, fp) < 0)
 -				return -1;
 -			if (fputs(st_el->prefix, fp) < 0 ||
 -			    fwrite(buf, p ? p - buf : count, 1, fp) != 1 ||
 -			    fputs(st_el->suffix, fp) < 0)
 -				return -1;
 -			if (st_el->color && *st_el->color
 -			    && fputs(GIT_COLOR_RESET, fp) < 0)
 -				return -1;
 +			const char *reset = st_el->color && *st_el->color ?
 +					    GIT_COLOR_RESET : NULL;
 +			if (st_el->color && *st_el->color)
 +				strbuf_addstr(&sb, st_el->color);
 +			strbuf_addstr(&sb, st_el->prefix);
 +			strbuf_add(&sb, buf, p ? p - buf : count);
 +			strbuf_addstr(&sb, st_el->suffix);
 +			if (reset)
 +				strbuf_addstr(&sb, reset);
  		}
  		if (!p)
 -			return 0;
 -		if (fputs(newline, fp) < 0)
 -			return -1;
 +			goto out;
 +
 +		strbuf_addstr(&sb, newline);
  		count -= p + 1 - buf;
  		buf = p + 1;
  		print = 1;
 +		if (count) {
 +			emit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,
 +					 sb.buf, sb.len, 0);
 +			strbuf_reset(&sb);
 +		}
  	}
 +
 +out:
 +	if (sb.len)
 +		emit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,
 +				 sb.buf, sb.len, 0);
 +	strbuf_release(&sb);
  	return 0;
  }
  
@@@ -1714,20 -987,24 +1717,20 @@@ static void fn_out_diff_words_aux(void 
  		fputs(line_prefix, diff_words->opt->file);
  	}
  	if (diff_words->current_plus != plus_begin) {
 -		fn_out_diff_words_write_helper(diff_words->opt->file,
 +		fn_out_diff_words_write_helper(diff_words->opt,
  				&style->ctx, style->newline,
  				plus_begin - diff_words->current_plus,
 -				diff_words->current_plus, line_prefix);
 -		if (*(plus_begin - 1) == '\n')
 -			fputs(line_prefix, diff_words->opt->file);
 +				diff_words->current_plus);
  	}
  	if (minus_begin != minus_end) {
 -		fn_out_diff_words_write_helper(diff_words->opt->file,
 +		fn_out_diff_words_write_helper(diff_words->opt,
  				&style->old, style->newline,
 -				minus_end - minus_begin, minus_begin,
 -				line_prefix);
 +				minus_end - minus_begin, minus_begin);
  	}
  	if (plus_begin != plus_end) {
 -		fn_out_diff_words_write_helper(diff_words->opt->file,
 +		fn_out_diff_words_write_helper(diff_words->opt,
  				&style->new, style->newline,
 -				plus_end - plus_begin, plus_begin,
 -				line_prefix);
 +				plus_end - plus_begin, plus_begin);
  	}
  
  	diff_words->current_plus = plus_end;
@@@ -1821,12 -1098,11 +1824,12 @@@ static void diff_words_show(struct diff
  
  	/* special case: only removal */
  	if (!diff_words->plus.text.size) {
 -		fputs(line_prefix, diff_words->opt->file);
 -		fn_out_diff_words_write_helper(diff_words->opt->file,
 +		emit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,
 +				 line_prefix, strlen(line_prefix), 0);
 +		fn_out_diff_words_write_helper(diff_words->opt,
  			&style->old, style->newline,
  			diff_words->minus.text.size,
 -			diff_words->minus.text.ptr, line_prefix);
 +			diff_words->minus.text.ptr);
  		diff_words->minus.text.size = 0;
  		return;
  	}
@@@ -1849,12 -1125,12 +1852,12 @@@
  	if (diff_words->current_plus != diff_words->plus.text.ptr +
  			diff_words->plus.text.size) {
  		if (color_words_output_graph_prefix(diff_words))
 -			fputs(line_prefix, diff_words->opt->file);
 -		fn_out_diff_words_write_helper(diff_words->opt->file,
 +			emit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,
 +					 line_prefix, strlen(line_prefix), 0);
 +		fn_out_diff_words_write_helper(diff_words->opt,
  			&style->ctx, style->newline,
  			diff_words->plus.text.ptr + diff_words->plus.text.size
 -			- diff_words->current_plus, diff_words->current_plus,
 -			line_prefix);
 +			- diff_words->current_plus, diff_words->current_plus);
  	}
  	diff_words->minus.text.size = diff_words->plus.text.size = 0;
  }
@@@ -1862,29 -1138,9 +1865,29 @@@
  /* In "color-words" mode, show word-diff of words accumulated in the buffer */
  static void diff_words_flush(struct emit_callback *ecbdata)
  {
 +	struct diff_options *wo = ecbdata->diff_words->opt;
 +
  	if (ecbdata->diff_words->minus.text.size ||
  	    ecbdata->diff_words->plus.text.size)
  		diff_words_show(ecbdata->diff_words);
 +
 +	if (wo->emitted_symbols) {
 +		struct diff_options *o = ecbdata->opt;
 +		struct emitted_diff_symbols *wol = wo->emitted_symbols;
 +		int i;
 +
 +		/*
 +		 * NEEDSWORK:
 +		 * Instead of appending each, concat all words to a line?
 +		 */
 +		for (i = 0; i < wol->nr; i++)
 +			append_emitted_diff_symbol(o, &wol->buf[i]);
 +
 +		for (i = 0; i < wol->nr; i++)
 +			free((void *)wol->buf[i].line);
 +
 +		wol->nr = 0;
 +	}
  }
  
  static void diff_filespec_load_driver(struct diff_filespec *one)
@@@ -1920,11 -1176,6 +1923,11 @@@ static void init_diff_words_data(struc
  		xcalloc(1, sizeof(struct diff_words_data));
  	ecbdata->diff_words->type = o->word_diff;
  	ecbdata->diff_words->opt = o;
 +
 +	if (orig_opts->emitted_symbols)
 +		o->emitted_symbols =
 +			xcalloc(1, sizeof(struct emitted_diff_symbols));
 +
  	if (!o->word_regex)
  		o->word_regex = userdiff_word_regex(one);
  	if (!o->word_regex)
@@@ -1959,7 -1210,6 +1962,7 @@@ static void free_diff_words_data(struc
  {
  	if (ecbdata->diff_words) {
  		diff_words_flush(ecbdata);
 +		free (ecbdata->diff_words->opt->emitted_symbols);
  		free (ecbdata->diff_words->opt);
  		free (ecbdata->diff_words->minus.text.ptr);
  		free (ecbdata->diff_words->minus.orig);
@@@ -1996,6 -1246,8 +1999,6 @@@ static unsigned long sane_truncate_line
  	unsigned long allot;
  	size_t l = len;
  
 -	if (ecb->truncate)
 -		return ecb->truncate(line, len);
  	cp = line;
  	allot = l;
  	while (0 < l) {
@@@ -2024,25 -1276,30 +2027,25 @@@ static void find_lno(const char *line, 
  static void fn_out_consume(void *priv, char *line, unsigned long len)
  {
  	struct emit_callback *ecbdata = priv;
 -	const char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);
 -	const char *context = diff_get_color(ecbdata->color_diff, DIFF_CONTEXT);
  	const char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);
  	struct diff_options *o = ecbdata->opt;
 -	const char *line_prefix = diff_line_prefix(o);
  
  	o->found_changes = 1;
  
  	if (ecbdata->header) {
 -		fprintf(o->file, "%s", ecbdata->header->buf);
 +		emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +				 ecbdata->header->buf, ecbdata->header->len, 0);
  		strbuf_reset(ecbdata->header);
  		ecbdata->header = NULL;
  	}
  
  	if (ecbdata->label_path[0]) {
 -		const char *name_a_tab, *name_b_tab;
 -
 -		name_a_tab = strchr(ecbdata->label_path[0], ' ') ? "\t" : "";
 -		name_b_tab = strchr(ecbdata->label_path[1], ' ') ? "\t" : "";
 -
 -		fprintf(o->file, "%s%s--- %s%s%s\n",
 -			line_prefix, meta, ecbdata->label_path[0], reset, name_a_tab);
 -		fprintf(o->file, "%s%s+++ %s%s%s\n",
 -			line_prefix, meta, ecbdata->label_path[1], reset, name_b_tab);
 +		emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,
 +				 ecbdata->label_path[0],
 +				 strlen(ecbdata->label_path[0]), 0);
 +		emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,
 +				 ecbdata->label_path[1],
 +				 strlen(ecbdata->label_path[1]), 0);
  		ecbdata->label_path[0] = ecbdata->label_path[1] = NULL;
  	}
  
@@@ -2058,13 -1315,12 +2061,13 @@@
  		len = sane_truncate_line(ecbdata, line, len);
  		find_lno(line, ecbdata);
  		emit_hunk_header(ecbdata, line, len);
 -		if (line[len-1] != '\n')
 -			putc('\n', o->file);
  		return;
  	}
  
  	if (ecbdata->diff_words) {
 +		enum diff_symbol s =
 +			ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN ?
 +			DIFF_SYMBOL_WORDS_PORCELAIN : DIFF_SYMBOL_WORDS;
  		if (line[0] == '-') {
  			diff_words_append(line, len,
  					  &ecbdata->diff_words->minus);
@@@ -2084,7 -1340,21 +2087,7 @@@
  			return;
  		}
  		diff_words_flush(ecbdata);
 -		if (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {
 -			emit_line(o, context, reset, line, len);
 -			fputs("~\n", o->file);
 -		} else {
 -			/*
 -			 * Skip the prefix character, if any.  With
 -			 * diff_suppress_blank_empty, there may be
 -			 * none.
 -			 */
 -			if (line[0] != '\n') {
 -			      line++;
 -			      len--;
 -			}
 -			emit_line(o, context, reset, line, len);
 -		}
 +		emit_diff_symbol(o, s, line, len, 0);
  		return;
  	}
  
@@@ -2105,8 -1375,8 +2108,8 @@@
  	default:
  		/* incomplete line at the end */
  		ecbdata->lno_in_preimage++;
 -		emit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),
 -			  reset, line, len);
 +		emit_diff_symbol(o, DIFF_SYMBOL_CONTEXT_INCOMPLETE,
 +				 line, len, 0);
  		break;
  	}
  }
@@@ -2251,14 -1521,20 +2254,14 @@@ static int scale_linear(int it, int wid
  	return 1 + (it * (width - 1) / max_change);
  }
  
 -static void show_name(FILE *file,
 -		      const char *prefix, const char *name, int len)
 -{
 -	fprintf(file, " %s%-*s |", prefix, len, name);
 -}
 -
 -static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)
 +static void show_graph(struct strbuf *out, char ch, int cnt,
 +		       const char *set, const char *reset)
  {
  	if (cnt <= 0)
  		return;
 -	fprintf(file, "%s", set);
 -	while (cnt--)
 -		putc(ch, file);
 -	fprintf(file, "%s", reset);
 +	strbuf_addstr(out, set);
 +	strbuf_addchars(out, ch, cnt);
 +	strbuf_addstr(out, reset);
  }
  
  static void fill_print_name(struct diffstat_file *file)
@@@ -2282,16 -1558,14 +2285,16 @@@
  	file->print_name = pname;
  }
  
 -int print_stat_summary(FILE *fp, int files, int insertions, int deletions)
 +static void print_stat_summary_inserts_deletes(struct diff_options *options,
 +		int files, int insertions, int deletions)
  {
  	struct strbuf sb = STRBUF_INIT;
 -	int ret;
  
  	if (!files) {
  		assert(insertions == 0 && deletions == 0);
 -		return fprintf(fp, "%s\n", " 0 files changed");
 +		emit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,
 +				 NULL, 0, 0);
 +		return;
  	}
  
  	strbuf_addf(&sb,
@@@ -2318,19 -1592,9 +2321,19 @@@
  			    deletions);
  	}
  	strbuf_addch(&sb, '\n');
 -	ret = fputs(sb.buf, fp);
 +	emit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
 +			 sb.buf, sb.len, 0);
  	strbuf_release(&sb);
 -	return ret;
 +}
 +
 +void print_stat_summary(FILE *fp, int files,
 +			int insertions, int deletions)
 +{
 +	struct diff_options o;
 +	memset(&o, 0, sizeof(o));
 +	o.file = fp;
 +
 +	print_stat_summary_inserts_deletes(&o, files, insertions, deletions);
  }
  
  static void show_stats(struct diffstat_t *data, struct diff_options *options)
@@@ -2340,13 -1604,13 +2343,13 @@@
  	int total_files = data->nr, count;
  	int width, name_width, graph_width, number_width = 0, bin_width = 0;
  	const char *reset, *add_c, *del_c;
 -	const char *line_prefix = "";
  	int extra_shown = 0;
 +	const char *line_prefix = diff_line_prefix(options);
 +	struct strbuf out = STRBUF_INIT;
  
  	if (data->nr == 0)
  		return;
  
 -	line_prefix = diff_line_prefix(options);
  	count = options->stat_count ? options->stat_count : data->nr;
  
  	reset = diff_get_color_opt(options, DIFF_RESET);
@@@ -2500,32 -1764,26 +2503,32 @@@
  		}
  
  		if (file->is_binary) {
 -			fprintf(options->file, "%s", line_prefix);
 -			show_name(options->file, prefix, name, len);
 -			fprintf(options->file, " %*s", number_width, "Bin");
 +			strbuf_addf(&out, " %s%-*s |", prefix, len, name);
 +			strbuf_addf(&out, " %*s", number_width, "Bin");
  			if (!added && !deleted) {
 -				putc('\n', options->file);
 +				strbuf_addch(&out, '\n');
 +				emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
 +						 out.buf, out.len, 0);
 +				strbuf_reset(&out);
  				continue;
  			}
 -			fprintf(options->file, " %s%"PRIuMAX"%s",
 +			strbuf_addf(&out, " %s%"PRIuMAX"%s",
  				del_c, deleted, reset);
 -			fprintf(options->file, " -> ");
 -			fprintf(options->file, "%s%"PRIuMAX"%s",
 +			strbuf_addstr(&out, " -> ");
 +			strbuf_addf(&out, "%s%"PRIuMAX"%s",
  				add_c, added, reset);
 -			fprintf(options->file, " bytes");
 -			fprintf(options->file, "\n");
 +			strbuf_addstr(&out, " bytes\n");
 +			emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
 +					 out.buf, out.len, 0);
 +			strbuf_reset(&out);
  			continue;
  		}
  		else if (file->is_unmerged) {
 -			fprintf(options->file, "%s", line_prefix);
 -			show_name(options->file, prefix, name, len);
 -			fprintf(options->file, " Unmerged\n");
 +			strbuf_addf(&out, " %s%-*s |", prefix, len, name);
 +			strbuf_addstr(&out, " Unmerged\n");
 +			emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
 +					 out.buf, out.len, 0);
 +			strbuf_reset(&out);
  			continue;
  		}
  
@@@ -2548,16 -1806,14 +2551,16 @@@
  				add = total - del;
  			}
  		}
 -		fprintf(options->file, "%s", line_prefix);
 -		show_name(options->file, prefix, name, len);
 -		fprintf(options->file, " %*"PRIuMAX"%s",
 +		strbuf_addf(&out, " %s%-*s |", prefix, len, name);
 +		strbuf_addf(&out, " %*"PRIuMAX"%s",
  			number_width, added + deleted,
  			added + deleted ? " " : "");
 -		show_graph(options->file, '+', add, add_c, reset);
 -		show_graph(options->file, '-', del, del_c, reset);
 -		fprintf(options->file, "\n");
 +		show_graph(&out, '+', add, add_c, reset);
 +		show_graph(&out, '-', del, del_c, reset);
 +		strbuf_addch(&out, '\n');
 +		emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
 +				 out.buf, out.len, 0);
 +		strbuf_reset(&out);
  	}
  
  	for (i = 0; i < data->nr; i++) {
@@@ -2578,14 -1834,11 +2581,14 @@@
  		if (i < count)
  			continue;
  		if (!extra_shown)
 -			fprintf(options->file, "%s ...\n", line_prefix);
 +			emit_diff_symbol(options,
 +					 DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
 +					 NULL, 0, 0);
  		extra_shown = 1;
  	}
 -	fprintf(options->file, "%s", line_prefix);
 -	print_stat_summary(options->file, total_files, adds, dels);
 +
 +	print_stat_summary_inserts_deletes(options, total_files, adds, dels);
 +	strbuf_release(&out);
  }
  
  static void show_shortstats(struct diffstat_t *data, struct diff_options *options)
@@@ -2597,7 -1850,7 +2600,7 @@@
  
  	for (i = 0; i < data->nr; i++) {
  		int added = data->files[i]->added;
 -		int deleted= data->files[i]->deleted;
 +		int deleted = data->files[i]->deleted;
  
  		if (data->files[i]->is_unmerged ||
  		    (!data->files[i]->is_interesting && (added + deleted == 0))) {
@@@ -2607,7 -1860,8 +2610,7 @@@
  			dels += deleted;
  		}
  	}
 -	fprintf(options->file, "%s", diff_line_prefix(options));
 -	print_stat_summary(options->file, total_files, adds, dels);
 +	print_stat_summary_inserts_deletes(options, total_files, adds, dels);
  }
  
  static void show_numstat(struct diffstat_t *data, struct diff_options *options)
@@@ -2971,8 -2225,8 +2974,8 @@@ static unsigned char *deflate_it(char *
  	return deflated;
  }
  
 -static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
 -				  const char *prefix)
 +static void emit_binary_diff_body(struct diff_options *o,
 +				  mmfile_t *one, mmfile_t *two)
  {
  	void *cp;
  	void *delta;
@@@ -3001,18 -2255,13 +3004,18 @@@
  	}
  
  	if (delta && delta_size < deflate_size) {
 -		fprintf(file, "%sdelta %lu\n", prefix, orig_size);
 +		char *s = xstrfmt("%lu", orig_size);
 +		emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,
 +				 s, strlen(s), 0);
 +		free(s);
  		free(deflated);
  		data = delta;
  		data_size = delta_size;
 -	}
 -	else {
 -		fprintf(file, "%sliteral %lu\n", prefix, two->size);
 +	} else {
 +		char *s = xstrfmt("%lu", two->size);
 +		emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,
 +				 s, strlen(s), 0);
 +		free(s);
  		free(delta);
  		data = deflated;
  		data_size = deflate_size;
@@@ -3021,9 -2270,8 +3024,9 @@@
  	/* emit data encoded in base85 */
  	cp = data;
  	while (data_size) {
 +		int len;
  		int bytes = (52 < data_size) ? 52 : data_size;
 -		char line[70];
 +		char line[71];
  		data_size -= bytes;
  		if (bytes <= 26)
  			line[0] = bytes + 'A' - 1;
@@@ -3031,24 -2279,20 +3034,24 @@@
  			line[0] = bytes - 26 + 'a' - 1;
  		encode_85(line + 1, cp, bytes);
  		cp = (char *) cp + bytes;
 -		fprintf(file, "%s", prefix);
 -		fputs(line, file);
 -		fputc('\n', file);
 +
 +		len = strlen(line);
 +		line[len++] = '\n';
 +		line[len] = '\0';
 +
 +		emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_BODY,
 +				 line, len, 0);
  	}
 -	fprintf(file, "%s\n", prefix);
 +	emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_FOOTER, NULL, 0, 0);
  	free(data);
  }
  
 -static void emit_binary_diff(FILE *file, mmfile_t *one, mmfile_t *two,
 -			     const char *prefix)
 +static void emit_binary_diff(struct diff_options *o,
 +			     mmfile_t *one, mmfile_t *two)
  {
 -	fprintf(file, "%sGIT binary patch\n", prefix);
 -	emit_binary_diff_body(file, one, two, prefix);
 -	emit_binary_diff_body(file, two, one, prefix);
 +	emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER, NULL, 0, 0);
 +	emit_binary_diff_body(o, one, two);
 +	emit_binary_diff_body(o, two, one);
  }
  
  int diff_filespec_is_binary(struct diff_filespec *one)
@@@ -3125,16 -2369,24 +3128,16 @@@ static void builtin_diff(const char *na
  	if (o->submodule_format == DIFF_SUBMODULE_LOG &&
  	    (!one->mode || S_ISGITLINK(one->mode)) &&
  	    (!two->mode || S_ISGITLINK(two->mode))) {
 -		const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
 -		const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
 -		show_submodule_summary(o->file, one->path ? one->path : two->path,
 -				line_prefix,
 +		show_submodule_summary(o, one->path ? one->path : two->path,
  				&one->oid, &two->oid,
 -				two->dirty_submodule,
 -				meta, del, add, reset);
 +				two->dirty_submodule);
  		return;
  	} else if (o->submodule_format == DIFF_SUBMODULE_INLINE_DIFF &&
  		   (!one->mode || S_ISGITLINK(one->mode)) &&
  		   (!two->mode || S_ISGITLINK(two->mode))) {
 -		const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
 -		const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
 -		show_submodule_inline_diff(o->file, one->path ? one->path : two->path,
 -				line_prefix,
 +		show_submodule_inline_diff(o, one->path ? one->path : two->path,
  				&one->oid, &two->oid,
 -				two->dirty_submodule,
 -				meta, del, add, reset, o);
 +				two->dirty_submodule);
  		return;
  	}
  
@@@ -3183,8 -2435,7 +3186,8 @@@
  		if (complete_rewrite &&
  		    (textconv_one || !diff_filespec_is_binary(one)) &&
  		    (textconv_two || !diff_filespec_is_binary(two))) {
 -			fprintf(o->file, "%s", header.buf);
 +			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +					 header.buf, header.len, 0);
  			strbuf_reset(&header);
  			emit_rewrite_diff(name_a, name_b, one, two,
  						textconv_one, textconv_two, o);
@@@ -3194,31 -2445,23 +3197,31 @@@
  	}
  
  	if (o->irreversible_delete && lbl[1][0] == '/') {
 -		fprintf(o->file, "%s", header.buf);
 +		emit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf,
 +				 header.len, 0);
  		strbuf_reset(&header);
  		goto free_ab_and_return;
  	} else if (!DIFF_OPT_TST(o, TEXT) &&
  	    ( (!textconv_one && diff_filespec_is_binary(one)) ||
  	      (!textconv_two && diff_filespec_is_binary(two)) )) {
 +		struct strbuf sb = STRBUF_INIT;
  		if (!one->data && !two->data &&
  		    S_ISREG(one->mode) && S_ISREG(two->mode) &&
  		    !DIFF_OPT_TST(o, BINARY)) {
  			if (!oidcmp(&one->oid, &two->oid)) {
  				if (must_show_header)
 -					fprintf(o->file, "%s", header.buf);
 +					emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +							 header.buf, header.len,
 +							 0);
  				goto free_ab_and_return;
  			}
 -			fprintf(o->file, "%s", header.buf);
 -			fprintf(o->file, "%sBinary files %s and %s differ\n",
 -				line_prefix, lbl[0], lbl[1]);
 +			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +					 header.buf, header.len, 0);
 +			strbuf_addf(&sb, "%sBinary files %s and %s differ\n",
 +				    diff_line_prefix(o), lbl[0], lbl[1]);
 +			emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
 +					 sb.buf, sb.len, 0);
 +			strbuf_release(&sb);
  			goto free_ab_and_return;
  		}
  		if (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)
@@@ -3227,21 -2470,16 +3230,21 @@@
  		if (mf1.size == mf2.size &&
  		    !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {
  			if (must_show_header)
 -				fprintf(o->file, "%s", header.buf);
 +				emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +						 header.buf, header.len, 0);
  			goto free_ab_and_return;
  		}
 -		fprintf(o->file, "%s", header.buf);
 +		emit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);
  		strbuf_reset(&header);
  		if (DIFF_OPT_TST(o, BINARY))
 -			emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
 -		else
 -			fprintf(o->file, "%sBinary files %s and %s differ\n",
 -				line_prefix, lbl[0], lbl[1]);
 +			emit_binary_diff(o, &mf1, &mf2);
 +		else {
 +			strbuf_addf(&sb, "%sBinary files %s and %s differ\n",
 +				    diff_line_prefix(o), lbl[0], lbl[1]);
 +			emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
 +					 sb.buf, sb.len, 0);
 +			strbuf_release(&sb);
 +		}
  		o->found_changes = 1;
  	} else {
  		/* Crazy xdl interfaces.. */
@@@ -3253,8 -2491,7 +3256,8 @@@
  		const struct userdiff_funcname *pe;
  
  		if (must_show_header) {
 -			fprintf(o->file, "%s", header.buf);
 +			emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +					 header.buf, header.len, 0);
  			strbuf_reset(&header);
  		}
  
@@@ -3723,6 -2960,7 +3726,6 @@@ static void prep_temp_blob(const char *
  			   const struct object_id *oid,
  			   int mode)
  {
 -	int fd;
  	struct strbuf buf = STRBUF_INIT;
  	struct strbuf template = STRBUF_INIT;
  	char *path_dup = xstrdup(path);
@@@ -3732,18 -2970,18 +3735,18 @@@
  	strbuf_addstr(&template, "XXXXXX_");
  	strbuf_addstr(&template, base);
  
 -	fd = mks_tempfile_ts(&temp->tempfile, template.buf, strlen(base) + 1);
 -	if (fd < 0)
 +	temp->tempfile = mks_tempfile_ts(template.buf, strlen(base) + 1);
 +	if (!temp->tempfile)
  		die_errno("unable to create temp-file");
  	if (convert_to_working_tree(path,
  			(const char *)blob, (size_t)size, &buf)) {
  		blob = buf.buf;
  		size = buf.len;
  	}
 -	if (write_in_full(fd, blob, size) != size)
 +	if (write_in_full(temp->tempfile->fd, blob, size) < 0 ||
 +	    close_tempfile_gently(temp->tempfile))
  		die_errno("unable to write temp-file");
 -	close_tempfile(&temp->tempfile);
 -	temp->name = get_tempfile_path(&temp->tempfile);
 +	temp->name = get_tempfile_path(temp->tempfile);
  	oid_to_hex_r(temp->hex, oid);
  	xsnprintf(temp->mode, sizeof(temp->mode), "%06o", mode);
  	strbuf_release(&buf);
@@@ -4011,7 -3249,7 +4014,7 @@@ static void diff_fill_oid_info(struct d
  			}
  			if (lstat(one->path, &st) < 0)
  				die_errno("stat '%s'", one->path);
 -			if (index_path(one->oid.hash, one->path, &st, 0))
 +			if (index_path(&one->oid, one->path, &st, 0))
  				die("cannot hash %s", one->path);
  		}
  	}
@@@ -4044,8 -3282,8 +4047,8 @@@ static void run_diff(struct diff_filepa
  	const char *other;
  	const char *attr_path;
  
 -	name  = p->one->path;
 -	other = (strcmp(name, p->two->path) ? p->two->path : NULL);
 +	name  = one->path;
 +	other = (strcmp(name, two->path) ? two->path : NULL);
  	attr_path = name;
  	if (o->prefix_length)
  		strip_prefix(o->prefix_length, &name, &other);
@@@ -4168,8 -3406,6 +4171,8 @@@ void diff_setup(struct diff_options *op
  		options->a_prefix = "a/";
  		options->b_prefix = "b/";
  	}
 +
 +	options->color_moved = diff_color_moved_default;
  }
  
  void diff_setup_done(struct diff_options *options)
@@@ -4279,9 -3515,6 +4282,9 @@@
  
  	if (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)
  		die(_("--follow requires exactly one pathspec"));
 +
 +	if (!options->use_color || external_diff())
 +		options->color_moved = 0;
  }
  
  static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)
@@@ -4706,19 -3939,7 +4709,19 @@@ int diff_opt_parse(struct diff_options 
  	}
  	else if (!strcmp(arg, "--no-color"))
  		options->use_color = 0;
 -	else if (!strcmp(arg, "--color-words")) {
 +	else if (!strcmp(arg, "--color-moved")) {
 +		if (diff_color_moved_default)
 +			options->color_moved = diff_color_moved_default;
 +		if (options->color_moved == COLOR_MOVED_NO)
 +			options->color_moved = COLOR_MOVED_DEFAULT;
 +	} else if (!strcmp(arg, "--no-color-moved"))
 +		options->color_moved = COLOR_MOVED_NO;
 +	else if (skip_prefix(arg, "--color-moved=", &arg)) {
 +		int cm = parse_color_moved(arg);
 +		if (cm < 0)
 +			die("bad --color-moved argument: %s", arg);
 +		options->color_moved = cm;
 +	} else if (!strcmp(arg, "--color-words")) {
  		options->use_color = 1;
  		options->word_diff = DIFF_WORDS_COLOR;
  	}
@@@ -5248,79 -4469,67 +5251,79 @@@ static void flush_one_pair(struct diff_
  	}
  }
  
 -static void show_file_mode_name(FILE *file, const char *newdelete, struct diff_filespec *fs)
 +static void show_file_mode_name(struct diff_options *opt, const char *newdelete, struct diff_filespec *fs)
  {
 +	struct strbuf sb = STRBUF_INIT;
  	if (fs->mode)
 -		fprintf(file, " %s mode %06o ", newdelete, fs->mode);
 +		strbuf_addf(&sb, " %s mode %06o ", newdelete, fs->mode);
  	else
 -		fprintf(file, " %s ", newdelete);
 -	write_name_quoted(fs->path, file, '\n');
 -}
 +		strbuf_addf(&sb, " %s ", newdelete);
  
 +	quote_c_style(fs->path, &sb, NULL, 0);
 +	strbuf_addch(&sb, '\n');
 +	emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
 +			 sb.buf, sb.len, 0);
 +	strbuf_release(&sb);
 +}
  
 -static void show_mode_change(FILE *file, struct diff_filepair *p, int show_name,
 -		const char *line_prefix)
 +static void show_mode_change(struct diff_options *opt, struct diff_filepair *p,
 +		int show_name)
  {
  	if (p->one->mode && p->two->mode && p->one->mode != p->two->mode) {
 -		fprintf(file, "%s mode change %06o => %06o%c", line_prefix, p->one->mode,
 -			p->two->mode, show_name ? ' ' : '\n');
 +		struct strbuf sb = STRBUF_INIT;
 +		strbuf_addf(&sb, " mode change %06o => %06o",
 +			    p->one->mode, p->two->mode);
  		if (show_name) {
 -			write_name_quoted(p->two->path, file, '\n');
 +			strbuf_addch(&sb, ' ');
 +			quote_c_style(p->two->path, &sb, NULL, 0);
  		}
 +		strbuf_addch(&sb, '\n');
 +		emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
 +				 sb.buf, sb.len, 0);
 +		strbuf_release(&sb);
  	}
  }
  
 -static void show_rename_copy(FILE *file, const char *renamecopy, struct diff_filepair *p,
 -			const char *line_prefix)
 +static void show_rename_copy(struct diff_options *opt, const char *renamecopy,
 +		struct diff_filepair *p)
  {
 +	struct strbuf sb = STRBUF_INIT;
  	char *names = pprint_rename(p->one->path, p->two->path);
 -
 -	fprintf(file, " %s %s (%d%%)\n", renamecopy, names, similarity_index(p));
 +	strbuf_addf(&sb, " %s %s (%d%%)\n",
 +			renamecopy, names, similarity_index(p));
  	free(names);
 -	show_mode_change(file, p, 0, line_prefix);
 +	emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
 +				 sb.buf, sb.len, 0);
 +	show_mode_change(opt, p, 0);
 +	strbuf_release(&sb);
  }
  
  static void diff_summary(struct diff_options *opt, struct diff_filepair *p)
  {
 -	FILE *file = opt->file;
 -	const char *line_prefix = diff_line_prefix(opt);
 -
  	switch(p->status) {
  	case DIFF_STATUS_DELETED:
 -		fputs(line_prefix, file);
 -		show_file_mode_name(file, "delete", p->one);
 +		show_file_mode_name(opt, "delete", p->one);
  		break;
  	case DIFF_STATUS_ADDED:
 -		fputs(line_prefix, file);
 -		show_file_mode_name(file, "create", p->two);
 +		show_file_mode_name(opt, "create", p->two);
  		break;
  	case DIFF_STATUS_COPIED:
 -		fputs(line_prefix, file);
 -		show_rename_copy(file, "copy", p, line_prefix);
 +		show_rename_copy(opt, "copy", p);
  		break;
  	case DIFF_STATUS_RENAMED:
 -		fputs(line_prefix, file);
 -		show_rename_copy(file, "rename", p, line_prefix);
 +		show_rename_copy(opt, "rename", p);
  		break;
  	default:
  		if (p->score) {
 -			fprintf(file, "%s rewrite ", line_prefix);
 -			write_name_quoted(p->two->path, file, ' ');
 -			fprintf(file, "(%d%%)\n", similarity_index(p));
 +			struct strbuf sb = STRBUF_INIT;
 +			strbuf_addstr(&sb, " rewrite ");
 +			quote_c_style(p->two->path, &sb, NULL, 0);
 +			strbuf_addf(&sb, " (%d%%)\n", similarity_index(p));
 +			emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
 +					 sb.buf, sb.len, 0);
 +			strbuf_release(&sb);
  		}
 -		show_mode_change(file, p, !p->score, line_prefix);
 +		show_mode_change(opt, p, !p->score);
  		break;
  	}
  }
@@@ -5525,51 -4734,6 +5528,51 @@@ void diff_warn_rename_limit(const char 
  		warning(_(rename_limit_advice), varname, needed);
  }
  
 +static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 +{
 +	int i;
 +	static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;
 +	struct diff_queue_struct *q = &diff_queued_diff;
 +
 +	if (WSEH_NEW & WS_RULE_MASK)
 +		die("BUG: WS rules bit mask overlaps with diff symbol flags");
 +
 +	if (o->color_moved)
 +		o->emitted_symbols = &esm;
 +
 +	for (i = 0; i < q->nr; i++) {
 +		struct diff_filepair *p = q->queue[i];
 +		if (check_pair_status(p))
 +			diff_flush_patch(p, o);
 +	}
 +
 +	if (o->emitted_symbols) {
 +		if (o->color_moved) {
 +			struct hashmap add_lines, del_lines;
 +
 +			hashmap_init(&del_lines,
 +				     (hashmap_cmp_fn)moved_entry_cmp, o, 0);
 +			hashmap_init(&add_lines,
 +				     (hashmap_cmp_fn)moved_entry_cmp, o, 0);
 +
 +			add_lines_to_move_detection(o, &add_lines, &del_lines);
 +			mark_color_as_moved(o, &add_lines, &del_lines);
 +			if (o->color_moved == COLOR_MOVED_ZEBRA_DIM)
 +				dim_moved_lines(o);
 +
 +			hashmap_free(&add_lines, 0);
 +			hashmap_free(&del_lines, 0);
 +		}
 +
 +		for (i = 0; i < esm.nr; i++)
 +			emit_diff_symbol_from_struct(o, &esm.buf[i]);
 +
 +		for (i = 0; i < esm.nr; i++)
 +			free((void *)esm.buf[i].line);
 +	}
 +	esm.nr = 0;
 +}
 +
  void diff_flush(struct diff_options *options)
  {
  	struct diff_queue_struct *q = &diff_queued_diff;
@@@ -5642,7 -4806,6 +5645,7 @@@
  			fclose(options->file);
  		options->file = xfopen("/dev/null", "w");
  		options->close_file = 1;
 +		options->color_moved = 0;
  		for (i = 0; i < q->nr; i++) {
  			struct diff_filepair *p = q->queue[i];
  			if (check_pair_status(p))
@@@ -5654,14 -4817,20 +5657,14 @@@
  
  	if (output_format & DIFF_FORMAT_PATCH) {
  		if (separator) {
 -			fprintf(options->file, "%s%c",
 -				diff_line_prefix(options),
 -				options->line_termination);
 -			if (options->stat_sep) {
 +			emit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);
 +			if (options->stat_sep)
  				/* attach patch instead of inline */
 -				fputs(options->stat_sep, options->file);
 -			}
 +				emit_diff_symbol(options, DIFF_SYMBOL_STAT_SEP,
 +						 NULL, 0, 0);
  		}
  
 -		for (i = 0; i < q->nr; i++) {
 -			struct diff_filepair *p = q->queue[i];
 -			if (check_pair_status(p))
 -				diff_flush_patch(p, options);
 -		}
 +		diff_flush_patch_all_file_pairs(options);
  	}
  
  	if (output_format & DIFF_FORMAT_CALLBACK)
diff --combined t/t6300-for-each-ref.sh
index 416ff7d0b8,09f2b7799d..3aa534933e
--- a/t/t6300-for-each-ref.sh
+++ b/t/t6300-for-each-ref.sh
@@@ -51,7 -51,6 +51,7 @@@ test_atom() 
  }
  
  test_atom head refname refs/heads/master
 +test_atom head refname: refs/heads/master
  test_atom head refname:short master
  test_atom head refname:lstrip=1 heads/master
  test_atom head refname:lstrip=2 master
@@@ -442,6 -441,11 +442,11 @@@ test_expect_success '--color can overri
  	test_cmp expected.color actual
  '
  
+ test_expect_success 'color.ui=always does not override tty check' '
+ 	git -c color.ui=always for-each-ref --format="$color_format" >actual &&
+ 	test_cmp expected.bare actual
+ '
+ 
  cat >expected <<\EOF
  heads/master
  tags/master
@@@ -605,104 -609,18 +610,104 @@@ test_expect_success 'do not dereferenc
  cat >trailers <<EOF
  Reviewed-by: A U Thor <author@example.com>
  Signed-off-by: A U Thor <author@example.com>
 +[ v2 updated patch description ]
 +Acked-by: A U Thor
 +  <author@example.com>
  EOF
  
 -test_expect_success 'basic atom: head contents:trailers' '
 +unfold () {
 +	perl -0pe 's/\n\s+/ /g'
 +}
 +
 +test_expect_success 'set up trailers for next test' '
  	echo "Some contents" > two &&
  	git add two &&
 -	git commit -F - <<-EOF &&
 +	git commit -F - <<-EOF
  	trailers: this commit message has trailers
  
  	Some message contents
  
  	$(cat trailers)
  	EOF
 +'
 +
 +test_expect_success '%(trailers:unfold) unfolds trailers' '
 +	git for-each-ref --format="%(trailers:unfold)" refs/heads/master >actual &&
 +	{
 +		unfold <trailers
 +		echo
 +	} >expect &&
 +	test_cmp expect actual
 +'
 +
 +test_expect_success '%(trailers:only) shows only "key: value" trailers' '
 +	git for-each-ref --format="%(trailers:only)" refs/heads/master >actual &&
 +	{
 +		grep -v patch.description <trailers &&
 +		echo
 +	} >expect &&
 +	test_cmp expect actual
 +'
 +
 +test_expect_success '%(trailers:only) and %(trailers:unfold) work together' '
 +	git for-each-ref --format="%(trailers:only,unfold)" refs/heads/master >actual &&
 +	git for-each-ref --format="%(trailers:unfold,only)" refs/heads/master >reverse &&
 +	test_cmp actual reverse &&
 +	{
 +		grep -v patch.description <trailers | unfold &&
 +		echo
 +	} >expect &&
 +	test_cmp expect actual
 +'
 +
 +test_expect_success '%(contents:trailers:unfold) unfolds trailers' '
 +	git for-each-ref --format="%(contents:trailers:unfold)" refs/heads/master >actual &&
 +	{
 +		unfold <trailers
 +		echo
 +	} >expect &&
 +	test_cmp expect actual
 +'
 +
 +test_expect_success '%(contents:trailers:only) shows only "key: value" trailers' '
 +	git for-each-ref --format="%(contents:trailers:only)" refs/heads/master >actual &&
 +	{
 +		grep -v patch.description <trailers &&
 +		echo
 +	} >expect &&
 +	test_cmp expect actual
 +'
 +
 +test_expect_success '%(contents:trailers:only) and %(contents:trailers:unfold) work together' '
 +	git for-each-ref --format="%(contents:trailers:only,unfold)" refs/heads/master >actual &&
 +	git for-each-ref --format="%(contents:trailers:unfold,only)" refs/heads/master >reverse &&
 +	test_cmp actual reverse &&
 +	{
 +		grep -v patch.description <trailers | unfold &&
 +		echo
 +	} >expect &&
 +	test_cmp expect actual
 +'
 +
 +test_expect_success '%(trailers) rejects unknown trailers arguments' '
 +	# error message cannot be checked under i18n
 +	cat >expect <<-EOF &&
 +	fatal: unknown %(trailers) argument: unsupported
 +	EOF
 +	test_must_fail git for-each-ref --format="%(trailers:unsupported)" 2>actual &&
 +	test_i18ncmp expect actual
 +'
 +
 +test_expect_success '%(contents:trailers) rejects unknown trailers arguments' '
 +	# error message cannot be checked under i18n
 +	cat >expect <<-EOF &&
 +	fatal: unknown %(trailers) argument: unsupported
 +	EOF
 +	test_must_fail git for-each-ref --format="%(contents:trailers:unsupported)" 2>actual &&
 +	test_i18ncmp expect actual
 +'
 +
 +test_expect_success 'basic atom: head contents:trailers' '
  	git for-each-ref --format="%(contents:trailers)" refs/heads/master >actual &&
  	sanitize_pgp <actual >actual.clean &&
  	# git for-each-ref ends with a blank line
diff --combined t/t7004-tag.sh
index 4e62c505fc,62aa322846..a9af2de996
--- a/t/t7004-tag.sh
+++ b/t/t7004-tag.sh
@@@ -1863,6 -1863,13 +1863,6 @@@ test_expect_success 'version sort with 
  	git tag -l --sort=version:refname
  '
  
 -run_with_limited_stack () {
 -	(ulimit -s 128 && "$@")
 -}
 -
 -test_lazy_prereq ULIMIT_STACK_SIZE 'run_with_limited_stack true'
 -
 -# we require ulimit, this excludes Windows
  test_expect_success ULIMIT_STACK_SIZE '--contains and --no-contains work in a deep repo' '
  	>expect &&
  	i=1 &&
@@@ -1918,6 -1925,12 +1918,12 @@@ test_expect_success '--color overrides 
  	test_cmp expect.color actual
  '
  
+ test_expect_success 'color.ui=always overrides auto-color' '
+ 	git -c color.ui=always tag $color_args >actual.raw &&
+ 	test_decode_color <actual.raw >actual &&
+ 	test_cmp expect.color actual
+ '
+ 
  test_expect_success 'setup --merged test tags' '
  	git tag mergetest-1 HEAD~2 &&
  	git tag mergetest-2 HEAD~1 &&