Merge branch 'jk/ref-filter-colors-fix'
authorJunio C Hamano <gitster@pobox.com>
Wed, 18 Oct 2017 01:19:08 +0000 (10:19 +0900)
committerJunio C Hamano <gitster@pobox.com>
Wed, 18 Oct 2017 01:19:08 +0000 (10:19 +0900)
This is the "theoretically more correct" approach of simply
stepping back to the state before plumbing commands started paying
attention to "color.ui" configuration variable.

Let's run with this one.

* jk/ref-filter-colors-fix:
tag: respect color.ui config
Revert "color: check color.ui in git_default_config()"
Revert "t6006: drop "always" color config tests"
Revert "color: make "always" the same as "auto" in config"

1  2 
Documentation/config.txt
builtin/branch.c
builtin/clean.c
builtin/grep.c
builtin/show-branch.c
builtin/tag.c
color.c
config.c
diff.c
t/t6300-for-each-ref.sh
t/t7004-tag.sh
diff --combined Documentation/config.txt
index b53c994d0a9f41da483aa22f66c7e308044eda1e,2271809d90a5855c40de64bf1952c58a48b0d386..1ac0ae6adb0460a3460084aef2302434e24ee6de
@@@ -776,12 -776,6 +776,12 @@@ core.commentChar:
  If set to "auto", `git-commit` would select a character that is not
  the beginning character of any line in existing commit messages.
  
 +core.filesRefLockTimeout::
 +      The length of time, in milliseconds, to retry when trying to
 +      lock an individual reference. Value 0 means not to retry at
 +      all; -1 means to try indefinitely. Default is 100 (i.e.,
 +      retry for 100ms).
 +
  core.packedRefsTimeout::
        The length of time, in milliseconds, to retry when trying to
        lock the `packed-refs` file. Value 0 means not to retry at
@@@ -1058,10 -1052,10 +1058,10 @@@ clean.requireForce:
  
  color.branch::
        A boolean to enable/disable color in the output of
-       linkgit:git-branch[1]. May be set to `false` (or `never`) to
-       disable color entirely, `auto` (or `true` or `always`) in which
-       case colors are used only when the output is to a terminal.  If
-       unset, then the value of `color.ui` is used (`auto` by default).
+       linkgit:git-branch[1]. May be set to `always`,
+       `false` (or `never`) or `auto` (or `true`), in which case colors are used
+       only when the output is to a terminal. If unset, then the
+       value of `color.ui` is used (`auto` by default).
  
  color.branch.<slot>::
        Use customized color for branch coloration. `<slot>` is one of
  
  color.diff::
        Whether to use ANSI escape sequences to add color to patches.
-       If this is set to `true` or `auto`, linkgit:git-diff[1],
+       If this is set to `always`, linkgit:git-diff[1],
        linkgit:git-log[1], and linkgit:git-show[1] will use color
-       when output is to the terminal. The value `always` is a
-       historical synonym for `auto`.  If unset, then the value of
-       `color.ui` is used (`auto` by default).
+       for all patches.  If it is set to `true` or `auto`, those
+       commands will only use color when output is to the terminal.
+       If unset, then the value of `color.ui` is used (`auto` by
+       default).
  +
  This does not affect linkgit:git-format-patch[1] or the
  'git-diff-{asterisk}' plumbing commands.  Can be overridden on the
  command line with the `--color[=<when>]` option.
  
 +diff.colorMoved::
 +      If set to either a valid `<mode>` or a true value, moved lines
 +      in a diff are colored differently, for details of valid modes
 +      see '--color-moved' in linkgit:git-diff[1]. If simply set to
 +      true the default color mode will be used. When set to false,
 +      moved lines are not colored.
 +
  color.diff.<slot>::
        Use customized color for diff colorization.  `<slot>` specifies
        which part of the patch to use the specified color, and is one
        of `context` (context text - `plain` is a historical synonym),
        `meta` (metainformation), `frag`
        (hunk header), 'func' (function in hunk header), `old` (removed lines),
 -      `new` (added lines), `commit` (commit headers), or `whitespace`
 -      (highlighting whitespace errors).
 +      `new` (added lines), `commit` (commit headers), `whitespace`
 +      (highlighting whitespace errors), `oldMoved` (deleted lines),
 +      `newMoved` (added lines), `oldMovedDimmed`, `oldMovedAlternative`,
 +      `oldMovedAlternativeDimmed`, `newMovedDimmed`, `newMovedAlternative`
 +      and `newMovedAlternativeDimmed` (See the '<mode>'
 +      setting of '--color-moved' in linkgit:git-diff[1] for details).
  
  color.decorate.<slot>::
        Use customized color for 'git log --decorate' output.  `<slot>` is one
@@@ -1140,12 -1124,12 +1141,12 @@@ color.grep.<slot>:
  --
  
  color.interactive::
-       When set to `true` or `auto`, use colors for interactive prompts
+       When set to `always`, always use colors for interactive prompts
        and displays (such as those used by "git-add --interactive" and
-       "git-clean --interactive") when the output is to the terminal.
-       When false (or `never`), never show colors. The value `always`
-       is a historical synonym for `auto`.  If unset, then the value of
-       `color.ui` is used (`auto` by default).
+       "git-clean --interactive"). When false (or `never`), never.
+       When set to `true` or `auto`, use colors only when the output is
+       to the terminal. If unset, then the value of `color.ui` is
+       used (`auto` by default).
  
  color.interactive.<slot>::
        Use customized color for 'git add --interactive' and 'git clean
@@@ -1192,10 -1176,10 +1193,10 @@@ color.ui:
        configuration to set a default for the `--color` option.  Set it
        to `false` or `never` if you prefer Git commands not to use
        color unless enabled explicitly with some other configuration
-       or the `--color` option. Set it to `true` or `auto` to enable
-       color when output is written to the terminal (this is also the
-       default since Git 1.8.4). The value `always` is a historical
-       synonym for `auto`.
+       or the `--color` option. Set it to `always` if you want all
+       output not intended for machine consumption to use color, to
+       `true` or `auto` (this is the default since Git 1.8.4) if you
+       want such output to use color when written to the terminal.
  
  column.ui::
        Specify whether supported commands should output in columns.
@@@ -1569,13 -1553,11 +1570,13 @@@ gc.<pattern>.reflogExpireUnreachable:
  gc.rerereResolved::
        Records of conflicted merge you resolved earlier are
        kept for this many days when 'git rerere gc' is run.
 +      You can also use more human-readable "1.month.ago", etc.
        The default is 60 days.  See linkgit:git-rerere[1].
  
  gc.rerereUnresolved::
        Records of conflicted merge you have not resolved are
        kept for this many days when 'git rerere gc' is run.
 +      You can also use more human-readable "1.month.ago", etc.
        The default is 15 days.  See linkgit:git-rerere[1].
  
  gitcvs.commitMsgAnnotation::
@@@ -3084,14 -3066,10 +3085,14 @@@ submodule.<name>.url:
        See linkgit:git-submodule[1] and linkgit:gitmodules[5] for details.
  
  submodule.<name>.update::
 -      The default update procedure for a submodule. This variable
 -      is populated by `git submodule init` from the
 -      linkgit:gitmodules[5] file. See description of 'update'
 -      command in linkgit:git-submodule[1].
 +      The method by which a submodule is updated by 'git submodule update',
 +      which is the only affected command, others such as
 +      'git checkout --recurse-submodules' are unaffected. It exists for
 +      historical reasons, when 'git submodule' was the only command to
 +      interact with submodules; settings like `submodule.active`
 +      and `pull.rebase` are more specific. It is populated by
 +      `git submodule init` from the linkgit:gitmodules[5] file.
 +      See description of 'update' command in linkgit:git-submodule[1].
  
  submodule.<name>.branch::
        The remote branch name for a submodule, used by `git submodule
diff --combined builtin/branch.c
index b67593288cf54084b569763ba2ef8e3b2cf9aa97,1969c7116cd1ae7cf3cf0f84eaf6ed333170113b..79dc9181fd6c0008d4b8c3b841f35cc1223d6d74
@@@ -28,7 -28,6 +28,7 @@@ static const char * const builtin_branc
        N_("git branch [<options>] [-l] [-f] <branch-name> [<start-point>]"),
        N_("git branch [<options>] [-r] (-d | -D) <branch-name>..."),
        N_("git branch [<options>] (-m | -M) [<old-branch>] <new-branch>"),
 +      N_("git branch [<options>] (-c | -C) [<old-branch>] <new-branch>"),
        N_("git branch [<options>] [-r | -a] [--points-at]"),
        N_("git branch [<options>] [-r | -a] [--format]"),
        NULL
@@@ -93,7 -92,7 +93,7 @@@ static int git_branch_config(const cha
                        return config_error_nonbool(var);
                return color_parse(value, branch_colors[slot]);
        }
-       return git_default_config(var, value, cb);
+       return git_color_default_config(var, value, cb);
  }
  
  static const char *branch_get_color(enum color_branch ix)
@@@ -217,7 -216,7 +217,7 @@@ static int delete_branches(int argc, co
                if (!head_rev)
                        die(_("Couldn't look up commit object for HEAD"));
        }
 -      for (i = 0; i < argc; i++, strbuf_release(&bname)) {
 +      for (i = 0; i < argc; i++, strbuf_reset(&bname)) {
                char *target = NULL;
                int flags = 0;
  
        }
  
        free(name);
 +      strbuf_release(&bname);
  
 -      return(ret);
 +      return ret;
  }
  
  static int calc_maxwidth(struct ref_array *refs, int remote_bonus)
@@@ -354,7 -352,7 +354,7 @@@ static char *build_format(struct ref_fi
                        strbuf_addf(&obname, "%%(objectname:short=%d)", filter->abbrev);
  
                strbuf_addf(&local, "%%(align:%d,left)%%(refname:lstrip=2)%%(end)", maxwidth);
 -              strbuf_addf(&local, "%s", branch_get_color(BRANCH_COLOR_RESET));
 +              strbuf_addstr(&local, branch_get_color(BRANCH_COLOR_RESET));
                strbuf_addf(&local, " %s ", obname.buf);
  
                if (filter->verbose > 1)
@@@ -458,19 -456,15 +458,19 @@@ static void reject_rebase_or_bisect_bra
        free_worktrees(worktrees);
  }
  
 -static void rename_branch(const char *oldname, const char *newname, int force)
 +static void copy_or_rename_branch(const char *oldname, const char *newname, int copy, int force)
  {
        struct strbuf oldref = STRBUF_INIT, newref = STRBUF_INIT, logmsg = STRBUF_INIT;
        struct strbuf oldsection = STRBUF_INIT, newsection = STRBUF_INIT;
        int recovery = 0;
        int clobber_head_ok;
  
 -      if (!oldname)
 -              die(_("cannot rename the current branch while not on any."));
 +      if (!oldname) {
 +              if (copy)
 +                      die(_("cannot copy the current branch while not on any."));
 +              else
 +                      die(_("cannot rename the current branch while not on any."));
 +      }
  
        if (strbuf_check_branch_ref(&oldref, oldname)) {
                /*
  
        reject_rebase_or_bisect_branch(oldref.buf);
  
 -      strbuf_addf(&logmsg, "Branch: renamed %s to %s",
 -               oldref.buf, newref.buf);
 +      if (copy)
 +              strbuf_addf(&logmsg, "Branch: copied %s to %s",
 +                          oldref.buf, newref.buf);
 +      else
 +              strbuf_addf(&logmsg, "Branch: renamed %s to %s",
 +                          oldref.buf, newref.buf);
  
 -      if (rename_ref(oldref.buf, newref.buf, logmsg.buf))
 +      if (!copy && rename_ref(oldref.buf, newref.buf, logmsg.buf))
                die(_("Branch rename failed"));
 +      if (copy && copy_existing_ref(oldref.buf, newref.buf, logmsg.buf))
 +              die(_("Branch copy failed"));
  
 -      if (recovery)
 -              warning(_("Renamed a misnamed branch '%s' away"), oldref.buf + 11);
 +      if (recovery) {
 +              if (copy)
 +                      warning(_("Copied a misnamed branch '%s' away"),
 +                              oldref.buf + 11);
 +              else
 +                      warning(_("Renamed a misnamed branch '%s' away"),
 +                              oldref.buf + 11);
 +      }
  
 -      if (replace_each_worktree_head_symref(oldref.buf, newref.buf, logmsg.buf))
 +      if (!copy &&
 +          replace_each_worktree_head_symref(oldref.buf, newref.buf, logmsg.buf))
                die(_("Branch renamed to %s, but HEAD is not updated!"), newname);
  
        strbuf_release(&logmsg);
        strbuf_release(&oldref);
        strbuf_addf(&newsection, "branch.%s", newref.buf + 11);
        strbuf_release(&newref);
 -      if (git_config_rename_section(oldsection.buf, newsection.buf) < 0)
 +      if (!copy && git_config_rename_section(oldsection.buf, newsection.buf) < 0)
                die(_("Branch is renamed, but update of config-file failed"));
 +      if (copy && strcmp(oldname, newname) && git_config_copy_section(oldsection.buf, newsection.buf) < 0)
 +              die(_("Branch is copied, but update of config-file failed"));
        strbuf_release(&oldsection);
        strbuf_release(&newsection);
  }
@@@ -565,7 -544,7 +565,7 @@@ static int edit_branch_description(cons
  
  int cmd_branch(int argc, const char **argv, const char *prefix)
  {
 -      int delete = 0, rename = 0, force = 0, list = 0;
 +      int delete = 0, rename = 0, copy = 0, force = 0, list = 0;
        int reflog = 0, edit_description = 0;
        int quiet = 0, unset_upstream = 0;
        const char *new_upstream = NULL;
                OPT__QUIET(&quiet, N_("suppress informational messages")),
                OPT_SET_INT('t', "track",  &track, N_("set up tracking mode (see git-pull(1))"),
                        BRANCH_TRACK_EXPLICIT),
 -              OPT_SET_INT( 0, "set-upstream",  &track, N_("change upstream info"),
 -                      BRANCH_TRACK_OVERRIDE),
 +              { OPTION_SET_INT, 0, "set-upstream", &track, NULL, N_("do not use"),
 +                      PARSE_OPT_NOARG | PARSE_OPT_HIDDEN, NULL, BRANCH_TRACK_OVERRIDE },
                OPT_STRING('u', "set-upstream-to", &new_upstream, N_("upstream"), N_("change the upstream info")),
                OPT_BOOL(0, "unset-upstream", &unset_upstream, N_("Unset the upstream info")),
                OPT__COLOR(&branch_use_color, N_("use colored output")),
                OPT_BIT('D', NULL, &delete, N_("delete branch (even if not merged)"), 2),
                OPT_BIT('m', "move", &rename, N_("move/rename a branch and its reflog"), 1),
                OPT_BIT('M', NULL, &rename, N_("move/rename a branch, even if target exists"), 2),
 +              OPT_BIT('c', "copy", &copy, N_("copy a branch and its reflog"), 1),
 +              OPT_BIT('C', NULL, &copy, N_("copy a branch, even if target exists"), 2),
                OPT_BOOL(0, "list", &list, N_("list branch names")),
                OPT_BOOL('l', "create-reflog", &reflog, N_("create the branch's reflog")),
                OPT_BOOL(0, "edit-description", &edit_description,
        argc = parse_options(argc, argv, prefix, options, builtin_branch_usage,
                             0);
  
 -      if (!delete && !rename && !edit_description && !new_upstream && !unset_upstream && argc == 0)
 +      if (!delete && !rename && !copy && !edit_description && !new_upstream && !unset_upstream && argc == 0)
                list = 1;
  
        if (filter.with_commit || filter.merge != REF_FILTER_MERGED_NONE || filter.points_at.nr ||
            filter.no_commit)
                list = 1;
  
 -      if (!!delete + !!rename + !!new_upstream +
 +      if (!!delete + !!rename + !!copy + !!new_upstream +
            list + unset_upstream > 1)
                usage_with_options(builtin_branch_usage, options);
  
        if (force) {
                delete *= 2;
                rename *= 2;
 +              copy *= 2;
        }
  
        if (delete) {
  
                if (edit_branch_description(branch_name))
                        return 1;
 +      } else if (copy) {
 +              if (!argc)
 +                      die(_("branch name required"));
 +              else if (argc == 1)
 +                      copy_or_rename_branch(head, argv[0], 1, copy > 1);
 +              else if (argc == 2)
 +                      copy_or_rename_branch(argv[0], argv[1], 1, copy > 1);
 +              else
 +                      die(_("too many branches for a copy operation"));
        } else if (rename) {
                if (!argc)
                        die(_("branch name required"));
                else if (argc == 1)
 -                      rename_branch(head, argv[0], rename > 1);
 +                      copy_or_rename_branch(head, argv[0], 0, rename > 1);
                else if (argc == 2)
 -                      rename_branch(argv[0], argv[1], rename > 1);
 +                      copy_or_rename_branch(argv[0], argv[1], 0, rename > 1);
                else
 -                      die(_("too many branches for a rename operation"));
 +                      die(_("too many arguments for a rename operation"));
        } else if (new_upstream) {
                struct branch *branch = branch_get(argv[0]);
  
                if (argc > 1)
 -                      die(_("too many branches to set new upstream"));
 +                      die(_("too many arguments to set new upstream"));
  
                if (!branch) {
                        if (!argc || !strcmp(argv[0], "HEAD"))
                struct strbuf buf = STRBUF_INIT;
  
                if (argc > 1)
 -                      die(_("too many branches to unset upstream"));
 +                      die(_("too many arguments to unset upstream"));
  
                if (!branch) {
                        if (!argc || !strcmp(argv[0], "HEAD"))
                strbuf_release(&buf);
        } else if (argc > 0 && argc <= 2) {
                struct branch *branch = branch_get(argv[0]);
 -              int branch_existed = 0, remote_tracking = 0;
 -              struct strbuf buf = STRBUF_INIT;
  
                if (!strcmp(argv[0], "HEAD"))
                        die(_("it does not make sense to create 'HEAD' manually"));
                        die(_("-a and -r options to 'git branch' do not make sense with a branch name"));
  
                if (track == BRANCH_TRACK_OVERRIDE)
 -                      fprintf(stderr, _("The --set-upstream flag is deprecated and will be removed. Consider using --track or --set-upstream-to\n"));
 -
 -              strbuf_addf(&buf, "refs/remotes/%s", branch->name);
 -              remote_tracking = ref_exists(buf.buf);
 -              strbuf_release(&buf);
 +                      die(_("the '--set-upstream' option is no longer supported. Please use '--track' or '--set-upstream-to' instead."));
  
 -              branch_existed = ref_exists(branch->refname);
                create_branch(argv[0], (argc == 2) ? argv[1] : head,
                              force, reflog, 0, quiet, track);
  
 -              /*
 -               * We only show the instructions if the user gave us
 -               * one branch which doesn't exist locally, but is the
 -               * name of a remote-tracking branch.
 -               */
 -              if (argc == 1 && track == BRANCH_TRACK_OVERRIDE &&
 -                  !branch_existed && remote_tracking) {
 -                      fprintf(stderr, _("\nIf you wanted to make '%s' track '%s', do this:\n\n"), head, branch->name);
 -                      fprintf(stderr, "    git branch -d %s\n", branch->name);
 -                      fprintf(stderr, "    git branch --set-upstream-to %s\n", branch->name);
 -              }
 -
        } else
                usage_with_options(builtin_branch_usage, options);
  
diff --combined builtin/clean.c
index 733b6d3745ee5780c697323372f1e78e28b8a117,057fc97fe4494338e6a85ac9f695563f1fcb9596..189e20628c07774089c5c925380b6b69af16fc7f
@@@ -33,6 -33,15 +33,6 @@@ static const char *msg_skip_git_dir = N
  static const char *msg_would_skip_git_dir = N_("Would skip repository %s\n");
  static const char *msg_warn_remove_failed = N_("failed to remove %s");
  
 -static int clean_use_color = -1;
 -static char clean_colors[][COLOR_MAXLEN] = {
 -      GIT_COLOR_RESET,
 -      GIT_COLOR_NORMAL,       /* PLAIN */
 -      GIT_COLOR_BOLD_BLUE,    /* PROMPT */
 -      GIT_COLOR_BOLD,         /* HEADER */
 -      GIT_COLOR_BOLD_RED,     /* HELP */
 -      GIT_COLOR_BOLD_RED,     /* ERROR */
 -};
  enum color_clean {
        CLEAN_COLOR_RESET = 0,
        CLEAN_COLOR_PLAIN = 1,
        CLEAN_COLOR_ERROR = 5
  };
  
 +static int clean_use_color = -1;
 +static char clean_colors[][COLOR_MAXLEN] = {
 +      [CLEAN_COLOR_ERROR] = GIT_COLOR_BOLD_RED,
 +      [CLEAN_COLOR_HEADER] = GIT_COLOR_BOLD,
 +      [CLEAN_COLOR_HELP] = GIT_COLOR_BOLD_RED,
 +      [CLEAN_COLOR_PLAIN] = GIT_COLOR_NORMAL,
 +      [CLEAN_COLOR_PROMPT] = GIT_COLOR_BOLD_BLUE,
 +      [CLEAN_COLOR_RESET] = GIT_COLOR_RESET,
 +};
 +
  #define MENU_OPTS_SINGLETON           01
  #define MENU_OPTS_IMMEDIATE           02
  #define MENU_OPTS_LIST_ONLY           04
@@@ -126,7 -125,8 +126,8 @@@ static int git_clean_config(const char 
                return 0;
        }
  
-       return git_default_config(var, value, cb);
+       /* inspect the color.ui config variable and others */
+       return git_color_default_config(var, value, cb);
  }
  
  static const char *clean_get_color(enum color_clean ix)
@@@ -167,7 -167,7 +168,7 @@@ static int remove_dirs(struct strbuf *p
                }
  
                *dir_gone = 0;
 -              return 0;
 +              goto out;
        }
  
        dir = opendir(path->buf);
                        warning_errno(_(msg_warn_remove_failed), quoted.buf);
                        *dir_gone = 0;
                }
 -              return res;
 +              ret = res;
 +              goto out;
        }
  
        strbuf_complete(path, '/');
                for (i = 0; i < dels.nr; i++)
                        printf(dry_run ?  _(msg_would_remove) : _(msg_remove), dels.items[i].string);
        }
 +out:
 +      strbuf_release(&quoted);
        string_list_clear(&dels, 0);
        return ret;
  }
diff --combined builtin/grep.c
index 19e23946ac4bfe53dd0f590fe26250b1f789778b,7e79eb1a754a5f604829e3e9e6d98d83712be9b6..2d65f27d01f3da772f7bad21f101fdc0af0102ed
@@@ -28,7 -28,13 +28,7 @@@ static char const * const grep_usage[] 
        NULL
  };
  
 -static const char *super_prefix;
  static int recurse_submodules;
 -static struct argv_array submodule_options = ARGV_ARRAY_INIT;
 -static const char *parent_basename;
 -
 -static int grep_submodule_launch(struct grep_opt *opt,
 -                               const struct grep_source *gs);
  
  #define GREP_NUM_THREADS_DEFAULT 8
  static int num_threads;
@@@ -180,7 -186,10 +180,7 @@@ static void *run(void *arg
                        break;
  
                opt->output_priv = w;
 -              if (w->source.type == GREP_SOURCE_SUBMODULE)
 -                      hit |= grep_submodule_launch(opt, &w->source);
 -              else
 -                      hit |= grep_source(opt, &w->source);
 +              hit |= grep_source(opt, &w->source);
                grep_source_clear_data(&w->source);
                work_done(w);
        }
@@@ -275,7 -284,7 +275,7 @@@ static int wait_all(void
  static int grep_cmd_config(const char *var, const char *value, void *cb)
  {
        int st = grep_config(var, value, cb);
-       if (git_default_config(var, value, cb) < 0)
+       if (git_color_default_config(var, value, cb) < 0)
                st = -1;
  
        if (!strcmp(var, "grep.threads")) {
@@@ -318,13 -327,21 +318,13 @@@ static int grep_oid(struct grep_opt *op
  {
        struct strbuf pathbuf = STRBUF_INIT;
  
 -      if (super_prefix) {
 -              strbuf_add(&pathbuf, filename, tree_name_len);
 -              strbuf_addstr(&pathbuf, super_prefix);
 -              strbuf_addstr(&pathbuf, filename + tree_name_len);
 +      if (opt->relative && opt->prefix_length) {
 +              quote_path_relative(filename + tree_name_len, opt->prefix, &pathbuf);
 +              strbuf_insert(&pathbuf, 0, filename, tree_name_len);
        } else {
                strbuf_addstr(&pathbuf, filename);
        }
  
 -      if (opt->relative && opt->prefix_length) {
 -              char *name = strbuf_detach(&pathbuf, NULL);
 -              quote_path_relative(name + tree_name_len, opt->prefix, &pathbuf);
 -              strbuf_insert(&pathbuf, 0, name, tree_name_len);
 -              free(name);
 -      }
 -
  #ifndef NO_PTHREADS
        if (num_threads) {
                add_work(opt, GREP_SOURCE_OID, pathbuf.buf, path, oid);
@@@ -349,10 -366,15 +349,10 @@@ static int grep_file(struct grep_opt *o
  {
        struct strbuf buf = STRBUF_INIT;
  
 -      if (super_prefix)
 -              strbuf_addstr(&buf, super_prefix);
 -      strbuf_addstr(&buf, filename);
 -
 -      if (opt->relative && opt->prefix_length) {
 -              char *name = strbuf_detach(&buf, NULL);
 -              quote_path_relative(name, opt->prefix, &buf);
 -              free(name);
 -      }
 +      if (opt->relative && opt->prefix_length)
 +              quote_path_relative(filename, opt->prefix, &buf);
 +      else
 +              strbuf_addstr(&buf, filename);
  
  #ifndef NO_PTHREADS
        if (num_threads) {
@@@ -399,89 -421,284 +399,89 @@@ static void run_pager(struct grep_opt *
                exit(status);
  }
  
 -static void compile_submodule_options(const struct grep_opt *opt,
 -                                    const char **argv,
 -                                    int cached, int untracked,
 -                                    int opt_exclude, int use_index,
 -                                    int pattern_type_arg)
 -{
 -      struct grep_pat *pattern;
 -
 -      if (recurse_submodules)
 -              argv_array_push(&submodule_options, "--recurse-submodules");
 -
 -      if (cached)
 -              argv_array_push(&submodule_options, "--cached");
 -      if (!use_index)
 -              argv_array_push(&submodule_options, "--no-index");
 -      if (untracked)
 -              argv_array_push(&submodule_options, "--untracked");
 -      if (opt_exclude > 0)
 -              argv_array_push(&submodule_options, "--exclude-standard");
 -
 -      if (opt->invert)
 -              argv_array_push(&submodule_options, "-v");
 -      if (opt->ignore_case)
 -              argv_array_push(&submodule_options, "-i");
 -      if (opt->word_regexp)
 -              argv_array_push(&submodule_options, "-w");
 -      switch (opt->binary) {
 -      case GREP_BINARY_NOMATCH:
 -              argv_array_push(&submodule_options, "-I");
 -              break;
 -      case GREP_BINARY_TEXT:
 -              argv_array_push(&submodule_options, "-a");
 -              break;
 -      default:
 -              break;
 -      }
 -      if (opt->allow_textconv)
 -              argv_array_push(&submodule_options, "--textconv");
 -      if (opt->max_depth != -1)
 -              argv_array_pushf(&submodule_options, "--max-depth=%d",
 -                               opt->max_depth);
 -      if (opt->linenum)
 -              argv_array_push(&submodule_options, "-n");
 -      if (!opt->pathname)
 -              argv_array_push(&submodule_options, "-h");
 -      if (!opt->relative)
 -              argv_array_push(&submodule_options, "--full-name");
 -      if (opt->name_only)
 -              argv_array_push(&submodule_options, "-l");
 -      if (opt->unmatch_name_only)
 -              argv_array_push(&submodule_options, "-L");
 -      if (opt->null_following_name)
 -              argv_array_push(&submodule_options, "-z");
 -      if (opt->count)
 -              argv_array_push(&submodule_options, "-c");
 -      if (opt->file_break)
 -              argv_array_push(&submodule_options, "--break");
 -      if (opt->heading)
 -              argv_array_push(&submodule_options, "--heading");
 -      if (opt->pre_context)
 -              argv_array_pushf(&submodule_options, "--before-context=%d",
 -                               opt->pre_context);
 -      if (opt->post_context)
 -              argv_array_pushf(&submodule_options, "--after-context=%d",
 -                               opt->post_context);
 -      if (opt->funcname)
 -              argv_array_push(&submodule_options, "-p");
 -      if (opt->funcbody)
 -              argv_array_push(&submodule_options, "-W");
 -      if (opt->all_match)
 -              argv_array_push(&submodule_options, "--all-match");
 -      if (opt->debug)
 -              argv_array_push(&submodule_options, "--debug");
 -      if (opt->status_only)
 -              argv_array_push(&submodule_options, "-q");
 -
 -      switch (pattern_type_arg) {
 -      case GREP_PATTERN_TYPE_BRE:
 -              argv_array_push(&submodule_options, "-G");
 -              break;
 -      case GREP_PATTERN_TYPE_ERE:
 -              argv_array_push(&submodule_options, "-E");
 -              break;
 -      case GREP_PATTERN_TYPE_FIXED:
 -              argv_array_push(&submodule_options, "-F");
 -              break;
 -      case GREP_PATTERN_TYPE_PCRE:
 -              argv_array_push(&submodule_options, "-P");
 -              break;
 -      case GREP_PATTERN_TYPE_UNSPECIFIED:
 -              break;
 -      default:
 -              die("BUG: Added a new grep pattern type without updating switch statement");
 -      }
 -
 -      for (pattern = opt->pattern_list; pattern != NULL;
 -           pattern = pattern->next) {
 -              switch (pattern->token) {
 -              case GREP_PATTERN:
 -                      argv_array_pushf(&submodule_options, "-e%s",
 -                                       pattern->pattern);
 -                      break;
 -              case GREP_AND:
 -              case GREP_OPEN_PAREN:
 -              case GREP_CLOSE_PAREN:
 -              case GREP_NOT:
 -              case GREP_OR:
 -                      argv_array_push(&submodule_options, pattern->pattern);
 -                      break;
 -              /* BODY and HEAD are not used by git-grep */
 -              case GREP_PATTERN_BODY:
 -              case GREP_PATTERN_HEAD:
 -                      break;
 -              }
 -      }
 -
 -      /*
 -       * Limit number of threads for child process to use.
 -       * This is to prevent potential fork-bomb behavior of git-grep as each
 -       * submodule process has its own thread pool.
 -       */
 -      argv_array_pushf(&submodule_options, "--threads=%d",
 -                       DIV_ROUND_UP(num_threads, 2));
 -
 -      /* Add Pathspecs */
 -      argv_array_push(&submodule_options, "--");
 -      for (; *argv; argv++)
 -              argv_array_push(&submodule_options, *argv);
 -}
 +static int grep_cache(struct grep_opt *opt, struct repository *repo,
 +                    const struct pathspec *pathspec, int cached);
 +static int grep_tree(struct grep_opt *opt, const struct pathspec *pathspec,
 +                   struct tree_desc *tree, struct strbuf *base, int tn_len,
 +                   int check_attr, struct repository *repo);
  
 -/*
 - * Launch child process to grep contents of a submodule
 - */
 -static int grep_submodule_launch(struct grep_opt *opt,
 -                               const struct grep_source *gs)
 +static int grep_submodule(struct grep_opt *opt, struct repository *superproject,
 +                        const struct pathspec *pathspec,
 +                        const struct object_id *oid,
 +                        const char *filename, const char *path)
  {
 -      struct child_process cp = CHILD_PROCESS_INIT;
 -      int status, i;
 -      const char *end_of_base;
 -      const char *name;
 -      struct strbuf child_output = STRBUF_INIT;
 -
 -      end_of_base = strchr(gs->name, ':');
 -      if (gs->identifier && end_of_base)
 -              name = end_of_base + 1;
 -      else
 -              name = gs->name;
 +      struct repository submodule;
 +      int hit;
  
 -      prepare_submodule_repo_env(&cp.env_array);
 -      argv_array_push(&cp.env_array, GIT_DIR_ENVIRONMENT);
 +      if (!is_submodule_active(superproject, path))
 +              return 0;
  
 -      if (opt->relative && opt->prefix_length)
 -              argv_array_pushf(&cp.env_array, "%s=%s",
 -                               GIT_TOPLEVEL_PREFIX_ENVIRONMENT,
 -                               opt->prefix);
 +      if (repo_submodule_init(&submodule, superproject, path))
 +              return 0;
  
 -      /* Add super prefix */
 -      argv_array_pushf(&cp.args, "--super-prefix=%s%s/",
 -                       super_prefix ? super_prefix : "",
 -                       name);
 -      argv_array_push(&cp.args, "grep");
 +      repo_read_gitmodules(&submodule);
  
        /*
 -       * Add basename of parent project
 -       * When performing grep on a tree object the filename is prefixed
 -       * with the object's name: 'tree-name:filename'.  In order to
 -       * provide uniformity of output we want to pass the name of the
 -       * parent project's object name to the submodule so the submodule can
 -       * prefix its output with the parent's name and not its own OID.
 +       * NEEDSWORK: This adds the submodule's object directory to the list of
 +       * alternates for the single in-memory object store.  This has some bad
 +       * consequences for memory (processed objects will never be freed) and
 +       * performance (this increases the number of pack files git has to pay
 +       * attention to, to the sum of the number of pack files in all the
 +       * repositories processed so far).  This can be removed once the object
 +       * store is no longer global and instead is a member of the repository
 +       * object.
         */
 -      if (gs->identifier && end_of_base)
 -              argv_array_pushf(&cp.args, "--parent-basename=%.*s",
 -                               (int) (end_of_base - gs->name),
 -                               gs->name);
 +      add_to_alternates_memory(submodule.objectdir);
  
 -      /* Add options */
 -      for (i = 0; i < submodule_options.argc; i++) {
 -              /*
 -               * If there is a tree identifier for the submodule, add the
 -               * rev after adding the submodule options but before the
 -               * pathspecs.  To do this we listen for the '--' and insert the
 -               * oid before pushing the '--' onto the child process argv
 -               * array.
 -               */
 -              if (gs->identifier &&
 -                  !strcmp("--", submodule_options.argv[i])) {
 -                      argv_array_push(&cp.args, oid_to_hex(gs->identifier));
 -              }
 +      if (oid) {
 +              struct object *object;
 +              struct tree_desc tree;
 +              void *data;
 +              unsigned long size;
 +              struct strbuf base = STRBUF_INIT;
  
 -              argv_array_push(&cp.args, submodule_options.argv[i]);
 -      }
 +              object = parse_object_or_die(oid, oid_to_hex(oid));
  
 -      cp.git_cmd = 1;
 -      cp.dir = gs->path;
 +              grep_read_lock();
 +              data = read_object_with_reference(object->oid.hash, tree_type,
 +                                                &size, NULL);
 +              grep_read_unlock();
  
 -      /*
 -       * Capture output to output buffer and check the return code from the
 -       * child process.  A '0' indicates a hit, a '1' indicates no hit and
 -       * anything else is an error.
 -       */
 -      status = capture_command(&cp, &child_output, 0);
 -      if (status && (status != 1)) {
 -              /* flush the buffer */
 -              write_or_die(1, child_output.buf, child_output.len);
 -              die("process for submodule '%s' failed with exit code: %d",
 -                  gs->name, status);
 -      }
 +              if (!data)
 +                      die(_("unable to read tree (%s)"), oid_to_hex(&object->oid));
  
 -      opt->output(opt, child_output.buf, child_output.len);
 -      strbuf_release(&child_output);
 -      /* invert the return code to make a hit equal to 1 */
 -      return !status;
 -}
 +              strbuf_addstr(&base, filename);
 +              strbuf_addch(&base, '/');
  
 -/*
 - * Prep grep structures for a submodule grep
 - * oid: the oid of the submodule or NULL if using the working tree
 - * filename: name of the submodule including tree name of parent
 - * path: location of the submodule
 - */
 -static int grep_submodule(struct grep_opt *opt, const struct object_id *oid,
 -                        const char *filename, const char *path)
 -{
 -      if (!is_submodule_active(the_repository, path))
 -              return 0;
 -      if (!is_submodule_populated_gently(path, NULL)) {
 -              /*
 -               * If searching history, check for the presence of the
 -               * submodule's gitdir before skipping the submodule.
 -               */
 -              if (oid) {
 -                      const struct submodule *sub =
 -                                      submodule_from_path(null_sha1, path);
 -                      if (sub)
 -                              path = git_path("modules/%s", sub->name);
 -
 -                      if (!(is_directory(path) && is_git_directory(path)))
 -                              return 0;
 -              } else {
 -                      return 0;
 -              }
 +              init_tree_desc(&tree, data, size);
 +              hit = grep_tree(opt, pathspec, &tree, &base, base.len,
 +                              object->type == OBJ_COMMIT, &submodule);
 +              strbuf_release(&base);
 +              free(data);
 +      } else {
 +              hit = grep_cache(opt, &submodule, pathspec, 1);
        }
  
 -#ifndef NO_PTHREADS
 -      if (num_threads) {
 -              add_work(opt, GREP_SOURCE_SUBMODULE, filename, path, oid);
 -              return 0;
 -      } else
 -#endif
 -      {
 -              struct grep_source gs;
 -              int hit;
 -
 -              grep_source_init(&gs, GREP_SOURCE_SUBMODULE,
 -                               filename, path, oid);
 -              hit = grep_submodule_launch(opt, &gs);
 -
 -              grep_source_clear(&gs);
 -              return hit;
 -      }
 +      repo_clear(&submodule);
 +      return hit;
  }
  
 -static int grep_cache(struct grep_opt *opt, const struct pathspec *pathspec,
 -                    int cached)
 +static int grep_cache(struct grep_opt *opt, struct repository *repo,
 +                    const struct pathspec *pathspec, int cached)
  {
        int hit = 0;
        int nr;
        struct strbuf name = STRBUF_INIT;
        int name_base_len = 0;
 -      if (super_prefix) {
 -              name_base_len = strlen(super_prefix);
 -              strbuf_addstr(&name, super_prefix);
 +      if (repo->submodule_prefix) {
 +              name_base_len = strlen(repo->submodule_prefix);
 +              strbuf_addstr(&name, repo->submodule_prefix);
        }
  
 -      read_cache();
 +      repo_read_index(repo);
  
 -      for (nr = 0; nr < active_nr; nr++) {
 -              const struct cache_entry *ce = active_cache[nr];
 +      for (nr = 0; nr < repo->index->cache_nr; nr++) {
 +              const struct cache_entry *ce = repo->index->cache[nr];
                strbuf_setlen(&name, name_base_len);
                strbuf_addstr(&name, ce->name);
  
                            ce_skip_worktree(ce)) {
                                if (ce_stage(ce) || ce_intent_to_add(ce))
                                        continue;
 -                              hit |= grep_oid(opt, &ce->oid, ce->name,
 -                                               0, ce->name);
 +                              hit |= grep_oid(opt, &ce->oid, name.buf,
 +                                               0, name.buf);
                        } else {
 -                              hit |= grep_file(opt, ce->name);
 +                              hit |= grep_file(opt, name.buf);
                        }
                } else if (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&
                           submodule_path_match(pathspec, name.buf, NULL)) {
 -                      hit |= grep_submodule(opt, NULL, ce->name, ce->name);
 +                      hit |= grep_submodule(opt, repo, pathspec, NULL, ce->name, ce->name);
                } else {
                        continue;
                }
                if (ce_stage(ce)) {
                        do {
                                nr++;
 -                      } while (nr < active_nr &&
 -                               !strcmp(ce->name, active_cache[nr]->name));
 +                      } while (nr < repo->index->cache_nr &&
 +                               !strcmp(ce->name, repo->index->cache[nr]->name));
                        nr--; /* compensate for loop control */
                }
                if (hit && opt->status_only)
  
  static int grep_tree(struct grep_opt *opt, const struct pathspec *pathspec,
                     struct tree_desc *tree, struct strbuf *base, int tn_len,
 -                   int check_attr)
 +                   int check_attr, struct repository *repo)
  {
        int hit = 0;
        enum interesting match = entry_not_interesting;
        int old_baselen = base->len;
        struct strbuf name = STRBUF_INIT;
        int name_base_len = 0;
 -      if (super_prefix) {
 -              strbuf_addstr(&name, super_prefix);
 +      if (repo->submodule_prefix) {
 +              strbuf_addstr(&name, repo->submodule_prefix);
                name_base_len = name.len;
        }
  
                        strbuf_addch(base, '/');
                        init_tree_desc(&sub, data, size);
                        hit |= grep_tree(opt, pathspec, &sub, base, tn_len,
 -                                       check_attr);
 +                                       check_attr, repo);
                        free(data);
                } else if (recurse_submodules && S_ISGITLINK(entry.mode)) {
 -                      hit |= grep_submodule(opt, entry.oid, base->buf,
 -                                            base->buf + tn_len);
 +                      hit |= grep_submodule(opt, repo, pathspec, entry.oid,
 +                                            base->buf, base->buf + tn_len);
                }
  
                strbuf_setlen(base, old_baselen);
  }
  
  static int grep_object(struct grep_opt *opt, const struct pathspec *pathspec,
 -                     struct object *obj, const char *name, const char *path)
 +                     struct object *obj, const char *name, const char *path,
 +                     struct repository *repo)
  {
        if (obj->type == OBJ_BLOB)
                return grep_oid(opt, &obj->oid, name, 0, path);
                if (!data)
                        die(_("unable to read tree (%s)"), oid_to_hex(&obj->oid));
  
 -              /* Use parent's name as base when recursing submodules */
 -              if (recurse_submodules && parent_basename)
 -                      name = parent_basename;
 -
                len = name ? strlen(name) : 0;
                strbuf_init(&base, PATH_MAX + len + 1);
                if (len) {
                }
                init_tree_desc(&tree, data, size);
                hit = grep_tree(opt, pathspec, &tree, &base, base.len,
 -                              obj->type == OBJ_COMMIT);
 +                              obj->type == OBJ_COMMIT, repo);
                strbuf_release(&base);
                free(data);
                return hit;
  }
  
  static int grep_objects(struct grep_opt *opt, const struct pathspec *pathspec,
 +                      struct repository *repo,
                        const struct object_array *list)
  {
        unsigned int i;
                /* load the gitmodules file for this rev */
                if (recurse_submodules) {
                        submodule_free();
 -                      gitmodules_config_sha1(real_obj->oid.hash);
 +                      gitmodules_config_oid(&real_obj->oid);
                }
 -              if (grep_object(opt, pathspec, real_obj, list->objects[i].name, list->objects[i].path)) {
 +              if (grep_object(opt, pathspec, real_obj, list->objects[i].name, list->objects[i].path,
 +                              repo)) {
                        hit = 1;
                        if (opt->status_only)
                                break;
@@@ -787,6 -1005,9 +787,6 @@@ int cmd_grep(int argc, const char **arg
                            N_("ignore files specified via '.gitignore'"), 1),
                OPT_BOOL(0, "recurse-submodules", &recurse_submodules,
                         N_("recursively search in each submodule")),
 -              OPT_STRING(0, "parent-basename", &parent_basename,
 -                         N_("basename"),
 -                         N_("prepend parent project's basename to output")),
                OPT_GROUP(""),
                OPT_BOOL('v', "invert-match", &opt.invert,
                        N_("show non-matching lines")),
        init_grep_defaults();
        git_config(grep_cmd_config, NULL);
        grep_init(&opt, prefix);
 -      super_prefix = get_super_prefix();
  
        /*
         * If there is no -- then the paths must exist in the working
                        break;
                }
  
 -              if (get_sha1_with_context(arg, GET_SHA1_RECORD_PATH,
 -                                        oid.hash, &oc)) {
 +              if (get_oid_with_context(arg, GET_OID_RECORD_PATH,
 +                                       &oid, &oc)) {
                        if (seen_dashdash)
                                die(_("unable to resolve revision: %s"), arg);
                        break;
        }
  #endif
  
 -      if (recurse_submodules) {
 -              gitmodules_config();
 -              compile_submodule_options(&opt, argv + i, cached, untracked,
 -                                        opt_exclude, use_index,
 -                                        pattern_type_arg);
 -      }
 -
        if (show_in_pager && (cached || list.nr))
                die(_("--open-files-in-pager only works on the worktree"));
  
                if (!cached)
                        setup_work_tree();
  
 -              hit = grep_cache(&opt, &pathspec, cached);
 +              hit = grep_cache(&opt, the_repository, &pathspec, cached);
        } else {
                if (cached)
                        die(_("both --cached and trees are given."));
 -              hit = grep_objects(&opt, &pathspec, &list);
 +
 +              hit = grep_objects(&opt, &pathspec, the_repository, &list);
        }
  
        if (num_threads)
diff --combined builtin/show-branch.c
index 84547d6fba07cb7770d94e80cda170c391d04a21,7073a3eb9769cae1f00e714a93528b33ca651d9c..6fa1f62a88ac2704abc6f274ed7a6170bd3a2f4c
@@@ -393,7 -393,7 +393,7 @@@ static int append_head_ref(const char *
        /* If both heads/foo and tags/foo exists, get_sha1 would
         * get confused.
         */
 -      if (get_sha1(refname + ofs, tmp.hash) || oidcmp(&tmp, oid))
 +      if (get_oid(refname + ofs, &tmp) || oidcmp(&tmp, oid))
                ofs = 5;
        return append_ref(refname + ofs, oid, 0);
  }
@@@ -408,7 -408,7 +408,7 @@@ static int append_remote_ref(const cha
        /* If both heads/foo and tags/foo exists, get_sha1 would
         * get confused.
         */
 -      if (get_sha1(refname + ofs, tmp.hash) || oidcmp(&tmp, oid))
 +      if (get_oid(refname + ofs, &tmp) || oidcmp(&tmp, oid))
                ofs = 5;
        return append_ref(refname + ofs, oid, 0);
  }
@@@ -514,7 -514,7 +514,7 @@@ static int show_independent(struct comm
  static void append_one_rev(const char *av)
  {
        struct object_id revkey;
 -      if (!get_sha1(av, revkey.hash)) {
 +      if (!get_oid(av, &revkey)) {
                append_ref(av, &revkey, 0);
                return;
        }
@@@ -554,7 -554,7 +554,7 @@@ static int git_show_branch_config(cons
                return 0;
        }
  
-       return git_default_config(var, value, cb);
+       return git_color_default_config(var, value, cb);
  }
  
  static int omit_in_dense(struct commit *commit, struct commit **rev, int n)
@@@ -808,7 -808,7 +808,7 @@@ int cmd_show_branch(int ac, const char 
                        die(Q_("cannot handle more than %d rev.",
                               "cannot handle more than %d revs.",
                               MAX_REVS), MAX_REVS);
 -              if (get_sha1(ref_name[num_rev], revkey.hash))
 +              if (get_oid(ref_name[num_rev], &revkey))
                        die(_("'%s' is not a valid ref."), ref_name[num_rev]);
                commit = lookup_commit_reference(&revkey);
                if (!commit)
diff --combined builtin/tag.c
index 695cb0778e2cad52d829030b00f91a4b230224e1,00382a56f564bf55de13ba712b8e728b3b1c9b86..b38329b593b595edad3efde82aaf1ecb90a73141
@@@ -113,7 -113,7 +113,7 @@@ static int verify_tag(const char *name
        if (format->format)
                flags = GPG_VERIFY_OMIT_STATUS;
  
 -      if (gpg_verify_tag(oid->hash, name, flags))
 +      if (gpg_verify_tag(oid, name, flags))
                return -1;
  
        if (format->format)
@@@ -158,7 -158,7 +158,7 @@@ static int git_tag_config(const char *v
  
        if (starts_with(var, "column."))
                return git_column_config(var, value, "tag", &colopts);
-       return git_default_config(var, value, cb);
+       return git_color_default_config(var, value, cb);
  }
  
  static void write_tag_body(int fd, const struct object_id *oid)
@@@ -553,10 -553,9 +553,10 @@@ int cmd_tag(int argc, const char **argv
        if (force && !is_null_oid(&prev) && oidcmp(&prev, &object))
                printf(_("Updated tag '%s' (was %s)\n"), tag, find_unique_abbrev(prev.hash, DEFAULT_ABBREV));
  
 -      strbuf_release(&err);
 -      strbuf_release(&buf);
 -      strbuf_release(&ref);
 -      strbuf_release(&reflog_msg);
 +      UNLEAK(buf);
 +      UNLEAK(ref);
 +      UNLEAK(reflog_msg);
 +      UNLEAK(msg);
 +      UNLEAK(err);
        return 0;
  }
diff --combined color.c
index 9c0dc823701d6aff0979659cd19ebb9c92f2eb8a,31b6207a00de42a386e98c5656209ea7a010abe4..9a9261ac164f503e2a9240806f4f44bedf9b8192
+++ b/color.c
@@@ -308,7 -308,7 +308,7 @@@ int git_config_colorbool(const char *va
                if (!strcasecmp(value, "never"))
                        return 0;
                if (!strcasecmp(value, "always"))
-                       return var ? GIT_COLOR_AUTO : 1;
+                       return 1;
                if (!strcasecmp(value, "auto"))
                        return GIT_COLOR_AUTO;
        }
@@@ -338,13 -338,6 +338,13 @@@ static int check_auto_color(void
  
  int want_color(int var)
  {
 +      /*
 +       * NEEDSWORK: This function is sometimes used from multiple threads, and
 +       * we end up using want_auto racily. That "should not matter" since
 +       * we always write the same value, but it's still wrong. This function
 +       * is listed in .tsan-suppressions for the time being.
 +       */
 +
        static int want_auto = -1;
  
        if (var < 0)
@@@ -368,6 -361,14 +368,14 @@@ int git_color_config(const char *var, c
        return 0;
  }
  
+ int git_color_default_config(const char *var, const char *value, void *cb)
+ {
+       if (git_color_config(var, value, cb) < 0)
+               return -1;
+       return git_default_config(var, value, cb);
+ }
  void color_print_strbuf(FILE *fp, const char *color, const struct strbuf *sb)
  {
        if (*color)
diff --combined config.c
index 4831c1273542706437139ccc379afb38e7ca3d3b,231f9a750b96deda8d62f5a5debb384f40481566..adb7d7a3e5ee164d24f1d2420677a8ad5b61e49f
+++ b/config.c
@@@ -16,7 -16,6 +16,6 @@@
  #include "string-list.h"
  #include "utf8.h"
  #include "dir.h"
- #include "color.h"
  
  struct config_source {
        struct config_source *prev;
@@@ -929,7 -928,7 +928,7 @@@ ssize_t git_config_ssize_t(const char *
        return ret;
  }
  
 -int git_parse_maybe_bool(const char *value)
 +static int git_parse_maybe_bool_text(const char *value)
  {
        if (!value)
                return 1;
        return -1;
  }
  
 -int git_config_maybe_bool(const char *name, const char *value)
 +int git_parse_maybe_bool(const char *value)
  {
 -      int v = git_parse_maybe_bool(value);
 +      int v = git_parse_maybe_bool_text(value);
        if (0 <= v)
                return v;
        if (git_parse_int(value, &v))
  
  int git_config_bool_or_int(const char *name, const char *value, int *is_bool)
  {
 -      int v = git_parse_maybe_bool(value);
 +      int v = git_parse_maybe_bool_text(value);
        if (0 <= v) {
                *is_bool = 1;
                return v;
@@@ -1351,9 -1350,6 +1350,6 @@@ int git_default_config(const char *var
        if (starts_with(var, "advice."))
                return git_default_advice_config(var, value);
  
-       if (git_color_config(var, value, dummy) < 0)
-               return -1;
        if (!strcmp(var, "pager.color") || !strcmp(var, "color.pager")) {
                pager_use_color = git_config_bool(var,value);
                return 0;
@@@ -1464,9 -1460,9 +1460,9 @@@ int git_config_from_mem(config_fn_t fn
        return do_config_from(&top, fn, data);
  }
  
 -int git_config_from_blob_sha1(config_fn_t fn,
 +int git_config_from_blob_oid(config_fn_t fn,
                              const char *name,
 -                            const unsigned char *sha1,
 +                            const struct object_id *oid,
                              void *data)
  {
        enum object_type type;
        unsigned long size;
        int ret;
  
 -      buf = read_sha1_file(sha1, &type, &size);
 +      buf = read_sha1_file(oid->hash, &type, &size);
        if (!buf)
                return error("unable to load config blob object '%s'", name);
        if (type != OBJ_BLOB) {
@@@ -1492,11 -1488,11 +1488,11 @@@ static int git_config_from_blob_ref(con
                                    const char *name,
                                    void *data)
  {
 -      unsigned char sha1[20];
 +      struct object_id oid;
  
 -      if (get_sha1(name, sha1) < 0)
 +      if (get_oid(name, &oid) < 0)
                return error("unable to resolve config blob '%s'", name);
 -      return git_config_from_blob_sha1(fn, name, sha1, data);
 +      return git_config_from_blob_oid(fn, name, &oid, data);
  }
  
  const char *git_etc_gitconfig(void)
@@@ -1719,19 -1715,17 +1715,19 @@@ static int configset_add_value(struct c
  }
  
  static int config_set_element_cmp(const void *unused_cmp_data,
 -                                const struct config_set_element *e1,
 -                                const struct config_set_element *e2,
 +                                const void *entry,
 +                                const void *entry_or_key,
                                  const void *unused_keydata)
  {
 +      const struct config_set_element *e1 = entry;
 +      const struct config_set_element *e2 = entry_or_key;
 +
        return strcmp(e1->key, e2->key);
  }
  
  void git_configset_init(struct config_set *cs)
  {
 -      hashmap_init(&cs->config_hash, (hashmap_cmp_fn)config_set_element_cmp,
 -                   NULL, 0);
 +      hashmap_init(&cs->config_hash, config_set_element_cmp, NULL, 0);
        cs->hash_initialized = 1;
        cs->list.nr = 0;
        cs->list.alloc = 0;
@@@ -1852,7 -1846,7 +1848,7 @@@ int git_configset_get_maybe_bool(struc
  {
        const char *value;
        if (!git_configset_get_value(cs, key, &value)) {
 -              *dest = git_config_maybe_bool(key, value);
 +              *dest = git_parse_maybe_bool(value);
                if (*dest == -1)
                        return -1;
                return 0;
@@@ -2059,23 -2053,6 +2055,23 @@@ int git_config_get_pathname(const char 
        return repo_config_get_pathname(the_repository, key, dest);
  }
  
 +/*
 + * Note: This function exists solely to maintain backward compatibility with
 + * 'fetch' and 'update_clone' storing configuration in '.gitmodules' and should
 + * NOT be used anywhere else.
 + *
 + * Runs the provided config function on the '.gitmodules' file found in the
 + * working directory.
 + */
 +void config_from_gitmodules(config_fn_t fn, void *data)
 +{
 +      if (the_repository->worktree) {
 +              char *file = repo_worktree_path(the_repository, GITMODULES_FILE);
 +              git_config_from_file(fn, file, data);
 +              free(file);
 +      }
 +}
 +
  int git_config_get_expiry(const char *key, const char **output)
  {
        int ret = git_config_get_string_const(key, output);
        return ret;
  }
  
 +int git_config_get_expiry_in_days(const char *key, timestamp_t *expiry, timestamp_t now)
 +{
 +      char *expiry_string;
 +      intmax_t days;
 +      timestamp_t when;
 +
 +      if (git_config_get_string(key, &expiry_string))
 +              return 1; /* no such thing */
 +
 +      if (git_parse_signed(expiry_string, &days, maximum_signed_value_of_type(int))) {
 +              const int scale = 86400;
 +              *expiry = now - days * scale;
 +              return 0;
 +      }
 +
 +      if (!parse_expiry_date(expiry_string, &when)) {
 +              *expiry = when;
 +              return 0;
 +      }
 +      return -1; /* thing exists but cannot be parsed */
 +}
 +
  int git_config_get_untracked_cache(void)
  {
        int val = -1;
@@@ -2200,7 -2155,7 +2196,7 @@@ static struct 
        size_t *offset;
        unsigned int offset_alloc;
        enum { START, SECTION_SEEN, SECTION_END_SEEN, KEY_SEEN } state;
 -      int seen;
 +      unsigned int seen;
  } store;
  
  static int matches(const char *key, const char *value)
@@@ -2292,10 -2247,10 +2288,10 @@@ static int write_error(const char *file
        return 4;
  }
  
 -static int store_write_section(int fd, const char *key)
 +static struct strbuf store_create_section(const char *key)
  {
        const char *dot;
 -      int i, success;
 +      int i;
        struct strbuf sb = STRBUF_INIT;
  
        dot = memchr(key, '.', store.baselen);
                strbuf_addf(&sb, "[%.*s]\n", store.baselen, key);
        }
  
 -      success = write_in_full(fd, sb.buf, sb.len) == sb.len;
 +      return sb;
 +}
 +
 +static ssize_t write_section(int fd, const char *key)
 +{
 +      struct strbuf sb = store_create_section(key);
 +      ssize_t ret;
 +
 +      ret = write_in_full(fd, sb.buf, sb.len) == sb.len;
        strbuf_release(&sb);
  
 -      return success;
 +      return ret;
  }
  
 -static int store_write_pair(int fd, const char *key, const char *value)
 +static ssize_t write_pair(int fd, const char *key, const char *value)
  {
 -      int i, success;
 +      int i;
 +      ssize_t ret;
        int length = strlen(key + store.baselen + 1);
        const char *quote = "";
        struct strbuf sb = STRBUF_INIT;
                case '"':
                case '\\':
                        strbuf_addch(&sb, '\\');
 +                      /* fallthrough */
                default:
                        strbuf_addch(&sb, value[i]);
                        break;
                }
        strbuf_addf(&sb, "%s\n", quote);
  
 -      success = write_in_full(fd, sb.buf, sb.len) == sb.len;
 +      ret = write_in_full(fd, sb.buf, sb.len);
        strbuf_release(&sb);
  
 -      return success;
 +      return ret;
  }
  
  static ssize_t find_beginning_of_line(const char *contents, size_t size,
@@@ -2455,7 -2400,7 +2451,7 @@@ int git_config_set_multivar_in_file_gen
  {
        int fd = -1, in_fd = -1;
        int ret;
 -      struct lock_file *lock = NULL;
 +      struct lock_file lock = LOCK_INIT;
        char *filename_buf = NULL;
        char *contents = NULL;
        size_t contents_sz;
         * The lock serves a purpose in addition to locking: the new
         * contents of .git/config will be written into it.
         */
 -      lock = xcalloc(1, sizeof(struct lock_file));
 -      fd = hold_lock_file_for_update(lock, config_filename, 0);
 +      fd = hold_lock_file_for_update(&lock, config_filename, 0);
        if (fd < 0) {
                error_errno("could not lock config file %s", config_filename);
                free(store.key);
                }
  
                store.key = (char *)key;
 -              if (!store_write_section(fd, key) ||
 -                  !store_write_pair(fd, key, value))
 +              if (write_section(fd, key) < 0 ||
 +                  write_pair(fd, key, value) < 0)
                        goto write_err_out;
        } else {
                struct stat st;
                close(in_fd);
                in_fd = -1;
  
 -              if (chmod(get_lock_file_path(lock), st.st_mode & 07777) < 0) {
 -                      error_errno("chmod on %s failed", get_lock_file_path(lock));
 +              if (chmod(get_lock_file_path(&lock), st.st_mode & 07777) < 0) {
 +                      error_errno("chmod on %s failed", get_lock_file_path(&lock));
                        ret = CONFIG_NO_WRITE;
                        goto out_free;
                }
                        /* write the first part of the config */
                        if (copy_end > copy_begin) {
                                if (write_in_full(fd, contents + copy_begin,
 -                                                copy_end - copy_begin) <
 -                                  copy_end - copy_begin)
 +                                                copy_end - copy_begin) < 0)
                                        goto write_err_out;
                                if (new_line &&
 -                                  write_str_in_full(fd, "\n") != 1)
 +                                  write_str_in_full(fd, "\n") < 0)
                                        goto write_err_out;
                        }
                        copy_begin = store.offset[i];
                /* write the pair (value == NULL means unset) */
                if (value != NULL) {
                        if (store.state == START) {
 -                              if (!store_write_section(fd, key))
 +                              if (write_section(fd, key) < 0)
                                        goto write_err_out;
                        }
 -                      if (!store_write_pair(fd, key, value))
 +                      if (write_pair(fd, key, value) < 0)
                                goto write_err_out;
                }
  
                /* write the rest of the config */
                if (copy_begin < contents_sz)
                        if (write_in_full(fd, contents + copy_begin,
 -                                        contents_sz - copy_begin) <
 -                          contents_sz - copy_begin)
 +                                        contents_sz - copy_begin) < 0)
                                goto write_err_out;
  
                munmap(contents, contents_sz);
                contents = NULL;
        }
  
 -      if (commit_lock_file(lock) < 0) {
 +      if (commit_lock_file(&lock) < 0) {
                error_errno("could not write config file %s", config_filename);
                ret = CONFIG_NO_WRITE;
 -              lock = NULL;
                goto out_free;
        }
  
 -      /*
 -       * lock is committed, so don't try to roll it back below.
 -       * NOTE: Since lockfile.c keeps a linked list of all created
 -       * lock_file structures, it isn't safe to free(lock).  It's
 -       * better to just leave it hanging around.
 -       */
 -      lock = NULL;
        ret = 0;
  
        /* Invalidate the config cache */
        git_config_clear();
  
  out_free:
 -      if (lock)
 -              rollback_lock_file(lock);
 +      rollback_lock_file(&lock);
        free(filename_buf);
        if (contents)
                munmap(contents, contents_sz);
        return ret;
  
  write_err_out:
 -      ret = write_error(get_lock_file_path(lock));
 +      ret = write_error(get_lock_file_path(&lock));
        goto out_free;
  
  }
@@@ -2750,8 -2707,8 +2746,8 @@@ static int section_name_is_ok(const cha
  }
  
  /* if new_name == NULL, the section is removed instead */
 -int git_config_rename_section_in_file(const char *config_filename,
 -                                    const char *old_name, const char *new_name)
 +static int git_config_copy_or_rename_section_in_file(const char *config_filename,
 +                                    const char *old_name, const char *new_name, int copy)
  {
        int ret = 0, remove = 0;
        char *filename_buf = NULL;
        char buf[1024];
        FILE *config_file = NULL;
        struct stat st;
 +      struct strbuf copystr = STRBUF_INIT;
  
        if (new_name && !section_name_is_ok(new_name)) {
                ret = error("invalid section name: %s", new_name);
        while (fgets(buf, sizeof(buf), config_file)) {
                int i;
                int length;
 +              int is_section = 0;
                char *output = buf;
                for (i = 0; buf[i] && isspace(buf[i]); i++)
                        ; /* do nothing */
                if (buf[i] == '[') {
                        /* it's a section */
 -                      int offset = section_name_match(&buf[i], old_name);
 +                      int offset;
 +                      is_section = 1;
 +
 +                      /*
 +                       * When encountering a new section under -c we
 +                       * need to flush out any section we're already
 +                       * coping and begin anew. There might be
 +                       * multiple [branch "$name"] sections.
 +                       */
 +                      if (copystr.len > 0) {
 +                              if (write_in_full(out_fd, copystr.buf, copystr.len) != copystr.len) {
 +                                      ret = write_error(get_lock_file_path(lock));
 +                                      goto out;
 +                              }
 +                              strbuf_reset(&copystr);
 +                      }
 +
 +                      offset = section_name_match(&buf[i], old_name);
                        if (offset > 0) {
                                ret++;
                                if (new_name == NULL) {
                                        continue;
                                }
                                store.baselen = strlen(new_name);
 -                              if (!store_write_section(out_fd, new_name)) {
 -                                      ret = write_error(get_lock_file_path(lock));
 -                                      goto out;
 -                              }
 -                              /*
 -                               * We wrote out the new section, with
 -                               * a newline, now skip the old
 -                               * section's length
 -                               */
 -                              output += offset + i;
 -                              if (strlen(output) > 0) {
 +                              if (!copy) {
 +                                      if (write_section(out_fd, new_name) < 0) {
 +                                              ret = write_error(get_lock_file_path(lock));
 +                                              goto out;
 +                                      }
                                        /*
 -                                       * More content means there's
 -                                       * a declaration to put on the
 -                                       * next line; indent with a
 -                                       * tab
 +                                       * We wrote out the new section, with
 +                                       * a newline, now skip the old
 +                                       * section's length
                                         */
 -                                      output -= 1;
 -                                      output[0] = '\t';
 +                                      output += offset + i;
 +                                      if (strlen(output) > 0) {
 +                                              /*
 +                                               * More content means there's
 +                                               * a declaration to put on the
 +                                               * next line; indent with a
 +                                               * tab
 +                                               */
 +                                              output -= 1;
 +                                              output[0] = '\t';
 +                                      }
 +                              } else {
 +                                      copystr = store_create_section(new_name);
                                }
                        }
                        remove = 0;
                if (remove)
                        continue;
                length = strlen(output);
 -              if (write_in_full(out_fd, output, length) != length) {
 +
 +              if (!is_section && copystr.len > 0) {
 +                      strbuf_add(&copystr, output, length);
 +              }
 +
 +              if (write_in_full(out_fd, output, length) < 0) {
                        ret = write_error(get_lock_file_path(lock));
                        goto out;
                }
        }
 +
 +      /*
 +       * Copy a trailing section at the end of the config, won't be
 +       * flushed by the usual "flush because we have a new section
 +       * logic in the loop above.
 +       */
 +      if (copystr.len > 0) {
 +              if (write_in_full(out_fd, copystr.buf, copystr.len) != copystr.len) {
 +                      ret = write_error(get_lock_file_path(lock));
 +                      goto out;
 +              }
 +              strbuf_reset(&copystr);
 +      }
 +
        fclose(config_file);
        config_file = NULL;
  commit_and_out:
@@@ -2899,30 -2814,11 +2895,30 @@@ out_no_rollback
        return ret;
  }
  
 +int git_config_rename_section_in_file(const char *config_filename,
 +                                    const char *old_name, const char *new_name)
 +{
 +      return git_config_copy_or_rename_section_in_file(config_filename,
 +                                       old_name, new_name, 0);
 +}
 +
  int git_config_rename_section(const char *old_name, const char *new_name)
  {
        return git_config_rename_section_in_file(NULL, old_name, new_name);
  }
  
 +int git_config_copy_section_in_file(const char *config_filename,
 +                                    const char *old_name, const char *new_name)
 +{
 +      return git_config_copy_or_rename_section_in_file(config_filename,
 +                                       old_name, new_name, 1);
 +}
 +
 +int git_config_copy_section(const char *old_name, const char *new_name)
 +{
 +      return git_config_copy_section_in_file(NULL, old_name, new_name);
 +}
 +
  /*
   * Call this to report error for your variable that should not
   * get a boolean value (i.e. "[my] var" means "true").
diff --combined diff.c
index d76bb937c1e49bc5cd3ecaf9f498a90543def822,85e714f6c68d24e11228b69d2511c49811c979b4..6fd288420bc55b5213ad6dfea4a4f411543d6dbf
--- 1/diff.c
--- 2/diff.c
+++ b/diff.c
  #include "userdiff.h"
  #include "submodule-config.h"
  #include "submodule.h"
 +#include "hashmap.h"
  #include "ll-merge.h"
  #include "string-list.h"
  #include "argv-array.h"
  #include "graph.h"
 +#include "packfile.h"
  
  #ifdef NO_FAST_WORKING_DIRECTORY
  #define FAST_WORKING_DIRECTORY 0
@@@ -34,7 -32,6 +34,7 @@@ static int diff_indent_heuristic = 1
  static int diff_rename_limit_default = 400;
  static int diff_suppress_blank_empty;
  static int diff_use_color_default = -1;
 +static int diff_color_moved_default;
  static int diff_context_default = 3;
  static int diff_interhunk_context_default;
  static const char *diff_word_regex_cfg;
@@@ -59,14 -56,6 +59,14 @@@ static char diff_colors[][COLOR_MAXLEN
        GIT_COLOR_YELLOW,       /* COMMIT */
        GIT_COLOR_BG_RED,       /* WHITESPACE */
        GIT_COLOR_NORMAL,       /* FUNCINFO */
 +      GIT_COLOR_BOLD_MAGENTA, /* OLD_MOVED */
 +      GIT_COLOR_BOLD_BLUE,    /* OLD_MOVED ALTERNATIVE */
 +      GIT_COLOR_FAINT,        /* OLD_MOVED_DIM */
 +      GIT_COLOR_FAINT_ITALIC, /* OLD_MOVED_ALTERNATIVE_DIM */
 +      GIT_COLOR_BOLD_CYAN,    /* NEW_MOVED */
 +      GIT_COLOR_BOLD_YELLOW,  /* NEW_MOVED ALTERNATIVE */
 +      GIT_COLOR_FAINT,        /* NEW_MOVED_DIM */
 +      GIT_COLOR_FAINT_ITALIC, /* NEW_MOVED_ALTERNATIVE_DIM */
  };
  
  static NORETURN void die_want_option(const char *option_name)
@@@ -92,22 -81,6 +92,22 @@@ static int parse_diff_color_slot(const 
                return DIFF_WHITESPACE;
        if (!strcasecmp(var, "func"))
                return DIFF_FUNCINFO;
 +      if (!strcasecmp(var, "oldmoved"))
 +              return DIFF_FILE_OLD_MOVED;
 +      if (!strcasecmp(var, "oldmovedalternative"))
 +              return DIFF_FILE_OLD_MOVED_ALT;
 +      if (!strcasecmp(var, "oldmoveddimmed"))
 +              return DIFF_FILE_OLD_MOVED_DIM;
 +      if (!strcasecmp(var, "oldmovedalternativedimmed"))
 +              return DIFF_FILE_OLD_MOVED_ALT_DIM;
 +      if (!strcasecmp(var, "newmoved"))
 +              return DIFF_FILE_NEW_MOVED;
 +      if (!strcasecmp(var, "newmovedalternative"))
 +              return DIFF_FILE_NEW_MOVED_ALT;
 +      if (!strcasecmp(var, "newmoveddimmed"))
 +              return DIFF_FILE_NEW_MOVED_DIM;
 +      if (!strcasecmp(var, "newmovedalternativedimmed"))
 +              return DIFF_FILE_NEW_MOVED_ALT_DIM;
        return -1;
  }
  
@@@ -256,44 -229,12 +256,44 @@@ int git_diff_heuristic_config(const cha
        return 0;
  }
  
 +static int parse_color_moved(const char *arg)
 +{
 +      switch (git_parse_maybe_bool(arg)) {
 +      case 0:
 +              return COLOR_MOVED_NO;
 +      case 1:
 +              return COLOR_MOVED_DEFAULT;
 +      default:
 +              break;
 +      }
 +
 +      if (!strcmp(arg, "no"))
 +              return COLOR_MOVED_NO;
 +      else if (!strcmp(arg, "plain"))
 +              return COLOR_MOVED_PLAIN;
 +      else if (!strcmp(arg, "zebra"))
 +              return COLOR_MOVED_ZEBRA;
 +      else if (!strcmp(arg, "default"))
 +              return COLOR_MOVED_DEFAULT;
 +      else if (!strcmp(arg, "dimmed_zebra"))
 +              return COLOR_MOVED_ZEBRA_DIM;
 +      else
 +              return error(_("color moved setting must be one of 'no', 'default', 'zebra', 'dimmed_zebra', 'plain'"));
 +}
 +
  int git_diff_ui_config(const char *var, const char *value, void *cb)
  {
        if (!strcmp(var, "diff.color") || !strcmp(var, "color.diff")) {
                diff_use_color_default = git_config_colorbool(var, value);
                return 0;
        }
 +      if (!strcmp(var, "diff.colormoved")) {
 +              int cm = parse_color_moved(value);
 +              if (cm < 0)
 +                      return -1;
 +              diff_color_moved_default = cm;
 +              return 0;
 +      }
        if (!strcmp(var, "diff.context")) {
                diff_context_default = git_config_int(var, value);
                if (diff_context_default < 0)
                return 0;
        }
  
+       if (git_color_config(var, value, cb) < 0)
+               return -1;
        return git_diff_basic_config(var, value, cb);
  }
  
@@@ -402,6 -346,9 +405,6 @@@ int git_diff_basic_config(const char *v
                return 0;
        }
  
 -      if (starts_with(var, "submodule."))
 -              return parse_submodule_config_option(var, value);
 -
        if (git_diff_heuristic_config(var, value, cb) < 0)
                return -1;
  
@@@ -459,9 -406,11 +462,9 @@@ static struct diff_tempfile 
         * If this diff_tempfile instance refers to a temporary file,
         * this tempfile object is used to manage its lifetime.
         */
 -      struct tempfile tempfile;
 +      struct tempfile *tempfile;
  } diff_temp[2];
  
 -typedef unsigned long (*sane_truncate_fn)(char *line, unsigned long len);
 -
  struct emit_callback {
        int color_diff;
        unsigned ws_rule;
        int blank_at_eof_in_postimage;
        int lno_in_preimage;
        int lno_in_postimage;
 -      sane_truncate_fn truncate;
        const char **label_path;
        struct diff_words_data *diff_words;
        struct diff_options *opt;
@@@ -610,737 -560,68 +613,737 @@@ static void emit_line(struct diff_optio
        emit_line_0(o, set, reset, line[0], line+1, len-1);
  }
  
 -static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
 +enum diff_symbol {
 +      DIFF_SYMBOL_BINARY_DIFF_HEADER,
 +      DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,
 +      DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,
 +      DIFF_SYMBOL_BINARY_DIFF_BODY,
 +      DIFF_SYMBOL_BINARY_DIFF_FOOTER,
 +      DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,
 +      DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
 +      DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
 +      DIFF_SYMBOL_STATS_LINE,
 +      DIFF_SYMBOL_WORD_DIFF,
 +      DIFF_SYMBOL_STAT_SEP,
 +      DIFF_SYMBOL_SUMMARY,
 +      DIFF_SYMBOL_SUBMODULE_ADD,
 +      DIFF_SYMBOL_SUBMODULE_DEL,
 +      DIFF_SYMBOL_SUBMODULE_UNTRACKED,
 +      DIFF_SYMBOL_SUBMODULE_MODIFIED,
 +      DIFF_SYMBOL_SUBMODULE_HEADER,
 +      DIFF_SYMBOL_SUBMODULE_ERROR,
 +      DIFF_SYMBOL_SUBMODULE_PIPETHROUGH,
 +      DIFF_SYMBOL_REWRITE_DIFF,
 +      DIFF_SYMBOL_BINARY_FILES,
 +      DIFF_SYMBOL_HEADER,
 +      DIFF_SYMBOL_FILEPAIR_PLUS,
 +      DIFF_SYMBOL_FILEPAIR_MINUS,
 +      DIFF_SYMBOL_WORDS_PORCELAIN,
 +      DIFF_SYMBOL_WORDS,
 +      DIFF_SYMBOL_CONTEXT,
 +      DIFF_SYMBOL_CONTEXT_INCOMPLETE,
 +      DIFF_SYMBOL_PLUS,
 +      DIFF_SYMBOL_MINUS,
 +      DIFF_SYMBOL_NO_LF_EOF,
 +      DIFF_SYMBOL_CONTEXT_FRAGINFO,
 +      DIFF_SYMBOL_CONTEXT_MARKER,
 +      DIFF_SYMBOL_SEPARATOR
 +};
 +/*
 + * Flags for content lines:
 + * 0..12 are whitespace rules
 + * 13-15 are WSEH_NEW | WSEH_OLD | WSEH_CONTEXT
 + * 16 is marking if the line is blank at EOF
 + */
 +#define DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF    (1<<16)
 +#define DIFF_SYMBOL_MOVED_LINE                        (1<<17)
 +#define DIFF_SYMBOL_MOVED_LINE_ALT            (1<<18)
 +#define DIFF_SYMBOL_MOVED_LINE_UNINTERESTING  (1<<19)
 +#define DIFF_SYMBOL_CONTENT_WS_MASK (WSEH_NEW | WSEH_OLD | WSEH_CONTEXT | WS_RULE_MASK)
 +
 +/*
 + * This struct is used when we need to buffer the output of the diff output.
 + *
 + * NEEDSWORK: Instead of storing a copy of the line, add an offset pointer
 + * into the pre/post image file. This pointer could be a union with the
 + * line pointer. By storing an offset into the file instead of the literal line,
 + * we can decrease the memory footprint for the buffered output. At first we
 + * may want to only have indirection for the content lines, but we could also
 + * enhance the state for emitting prefabricated lines, e.g. the similarity
 + * score line or hunk/file headers would only need to store a number or path
 + * and then the output can be constructed later on depending on state.
 + */
 +struct emitted_diff_symbol {
 +      const char *line;
 +      int len;
 +      int flags;
 +      enum diff_symbol s;
 +};
 +#define EMITTED_DIFF_SYMBOL_INIT {NULL}
 +
 +struct emitted_diff_symbols {
 +      struct emitted_diff_symbol *buf;
 +      int nr, alloc;
 +};
 +#define EMITTED_DIFF_SYMBOLS_INIT {NULL, 0, 0}
 +
 +static void append_emitted_diff_symbol(struct diff_options *o,
 +                                     struct emitted_diff_symbol *e)
  {
 -      if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
 -            ecbdata->blank_at_eof_in_preimage &&
 -            ecbdata->blank_at_eof_in_postimage &&
 -            ecbdata->blank_at_eof_in_preimage <= ecbdata->lno_in_preimage &&
 -            ecbdata->blank_at_eof_in_postimage <= ecbdata->lno_in_postimage))
 -              return 0;
 -      return ws_blank_line(line, len, ecbdata->ws_rule);
 +      struct emitted_diff_symbol *f;
 +
 +      ALLOC_GROW(o->emitted_symbols->buf,
 +                 o->emitted_symbols->nr + 1,
 +                 o->emitted_symbols->alloc);
 +      f = &o->emitted_symbols->buf[o->emitted_symbols->nr++];
 +
 +      memcpy(f, e, sizeof(struct emitted_diff_symbol));
 +      f->line = e->line ? xmemdupz(e->line, e->len) : NULL;
  }
  
 -static void emit_line_checked(const char *reset,
 -                            struct emit_callback *ecbdata,
 -                            const char *line, int len,
 -                            enum color_diff color,
 -                            unsigned ws_error_highlight,
 -                            char sign)
 +struct moved_entry {
 +      struct hashmap_entry ent;
 +      const struct emitted_diff_symbol *es;
 +      struct moved_entry *next_line;
 +};
 +
 +static int next_byte(const char **cp, const char **endp,
 +                   const struct diff_options *diffopt)
 +{
 +      int retval;
 +
 +      if (*cp > *endp)
 +              return -1;
 +
 +      if (isspace(**cp)) {
 +              if (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_CHANGE)) {
 +                      while (*cp < *endp && isspace(**cp))
 +                              (*cp)++;
 +                      /*
 +                       * After skipping a couple of whitespaces,
 +                       * we still have to account for one space.
 +                       */
 +                      return (int)' ';
 +              }
 +
 +              if (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE)) {
 +                      while (*cp < *endp && isspace(**cp))
 +                              (*cp)++;
 +                      /* return the first non-ws character via the usual below */
 +              }
 +      }
 +
 +      retval = (unsigned char)(**cp);
 +      (*cp)++;
 +      return retval;
 +}
 +
 +static int moved_entry_cmp(const struct diff_options *diffopt,
 +                         const struct moved_entry *a,
 +                         const struct moved_entry *b,
 +                         const void *keydata)
 +{
 +      const char *ap = a->es->line, *ae = a->es->line + a->es->len;
 +      const char *bp = b->es->line, *be = b->es->line + b->es->len;
 +
 +      if (!(diffopt->xdl_opts & XDF_WHITESPACE_FLAGS))
 +              return a->es->len != b->es->len  || memcmp(ap, bp, a->es->len);
 +
 +      if (DIFF_XDL_TST(diffopt, IGNORE_WHITESPACE_AT_EOL)) {
 +              while (ae > ap && isspace(*ae))
 +                      ae--;
 +              while (be > bp && isspace(*be))
 +                      be--;
 +      }
 +
 +      while (1) {
 +              int ca, cb;
 +              ca = next_byte(&ap, &ae, diffopt);
 +              cb = next_byte(&bp, &be, diffopt);
 +              if (ca != cb)
 +                      return 1;
 +              if (ca < 0)
 +                      return 0;
 +      }
 +}
 +
 +static unsigned get_string_hash(struct emitted_diff_symbol *es, struct diff_options *o)
 +{
 +      if (o->xdl_opts & XDF_WHITESPACE_FLAGS) {
 +              static struct strbuf sb = STRBUF_INIT;
 +              const char *ap = es->line, *ae = es->line + es->len;
 +              int c;
 +
 +              strbuf_reset(&sb);
 +              while (ae > ap && isspace(*ae))
 +                      ae--;
 +              while ((c = next_byte(&ap, &ae, o)) > 0)
 +                      strbuf_addch(&sb, c);
 +
 +              return memhash(sb.buf, sb.len);
 +      } else {
 +              return memhash(es->line, es->len);
 +      }
 +}
 +
 +static struct moved_entry *prepare_entry(struct diff_options *o,
 +                                       int line_no)
 +{
 +      struct moved_entry *ret = xmalloc(sizeof(*ret));
 +      struct emitted_diff_symbol *l = &o->emitted_symbols->buf[line_no];
 +
 +      ret->ent.hash = get_string_hash(l, o);
 +      ret->es = l;
 +      ret->next_line = NULL;
 +
 +      return ret;
 +}
 +
 +static void add_lines_to_move_detection(struct diff_options *o,
 +                                      struct hashmap *add_lines,
 +                                      struct hashmap *del_lines)
 +{
 +      struct moved_entry *prev_line = NULL;
 +
 +      int n;
 +      for (n = 0; n < o->emitted_symbols->nr; n++) {
 +              struct hashmap *hm;
 +              struct moved_entry *key;
 +
 +              switch (o->emitted_symbols->buf[n].s) {
 +              case DIFF_SYMBOL_PLUS:
 +                      hm = add_lines;
 +                      break;
 +              case DIFF_SYMBOL_MINUS:
 +                      hm = del_lines;
 +                      break;
 +              default:
 +                      prev_line = NULL;
 +                      continue;
 +              }
 +
 +              key = prepare_entry(o, n);
 +              if (prev_line && prev_line->es->s == o->emitted_symbols->buf[n].s)
 +                      prev_line->next_line = key;
 +
 +              hashmap_add(hm, key);
 +              prev_line = key;
 +      }
 +}
 +
 +static int shrink_potential_moved_blocks(struct moved_entry **pmb,
 +                                       int pmb_nr)
 +{
 +      int lp, rp;
 +
 +      /* Shrink the set of potential block to the remaining running */
 +      for (lp = 0, rp = pmb_nr - 1; lp <= rp;) {
 +              while (lp < pmb_nr && pmb[lp])
 +                      lp++;
 +              /* lp points at the first NULL now */
 +
 +              while (rp > -1 && !pmb[rp])
 +                      rp--;
 +              /* rp points at the last non-NULL */
 +
 +              if (lp < pmb_nr && rp > -1 && lp < rp) {
 +                      pmb[lp] = pmb[rp];
 +                      pmb[rp] = NULL;
 +                      rp--;
 +                      lp++;
 +              }
 +      }
 +
 +      /* Remember the number of running sets */
 +      return rp + 1;
 +}
 +
 +/*
 + * If o->color_moved is COLOR_MOVED_PLAIN, this function does nothing.
 + *
 + * Otherwise, if the last block has fewer alphanumeric characters than
 + * COLOR_MOVED_MIN_ALNUM_COUNT, unset DIFF_SYMBOL_MOVED_LINE on all lines in
 + * that block.
 + *
 + * The last block consists of the (n - block_length)'th line up to but not
 + * including the nth line.
 + *
 + * NEEDSWORK: This uses the same heuristic as blame_entry_score() in blame.c.
 + * Think of a way to unify them.
 + */
 +static void adjust_last_block(struct diff_options *o, int n, int block_length)
 +{
 +      int i, alnum_count = 0;
 +      if (o->color_moved == COLOR_MOVED_PLAIN)
 +              return;
 +      for (i = 1; i < block_length + 1; i++) {
 +              const char *c = o->emitted_symbols->buf[n - i].line;
 +              for (; *c; c++) {
 +                      if (!isalnum(*c))
 +                              continue;
 +                      alnum_count++;
 +                      if (alnum_count >= COLOR_MOVED_MIN_ALNUM_COUNT)
 +                              return;
 +              }
 +      }
 +      for (i = 1; i < block_length + 1; i++)
 +              o->emitted_symbols->buf[n - i].flags &= ~DIFF_SYMBOL_MOVED_LINE;
 +}
 +
 +/* Find blocks of moved code, delegate actual coloring decision to helper */
 +static void mark_color_as_moved(struct diff_options *o,
 +                              struct hashmap *add_lines,
 +                              struct hashmap *del_lines)
 +{
 +      struct moved_entry **pmb = NULL; /* potentially moved blocks */
 +      int pmb_nr = 0, pmb_alloc = 0;
 +      int n, flipped_block = 1, block_length = 0;
 +
 +
 +      for (n = 0; n < o->emitted_symbols->nr; n++) {
 +              struct hashmap *hm = NULL;
 +              struct moved_entry *key;
 +              struct moved_entry *match = NULL;
 +              struct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];
 +              int i;
 +
 +              switch (l->s) {
 +              case DIFF_SYMBOL_PLUS:
 +                      hm = del_lines;
 +                      key = prepare_entry(o, n);
 +                      match = hashmap_get(hm, key, o);
 +                      free(key);
 +                      break;
 +              case DIFF_SYMBOL_MINUS:
 +                      hm = add_lines;
 +                      key = prepare_entry(o, n);
 +                      match = hashmap_get(hm, key, o);
 +                      free(key);
 +                      break;
 +              default:
 +                      flipped_block = 1;
 +              }
 +
 +              if (!match) {
 +                      adjust_last_block(o, n, block_length);
 +                      pmb_nr = 0;
 +                      block_length = 0;
 +                      continue;
 +              }
 +
 +              l->flags |= DIFF_SYMBOL_MOVED_LINE;
 +
 +              if (o->color_moved == COLOR_MOVED_PLAIN)
 +                      continue;
 +
 +              /* Check any potential block runs, advance each or nullify */
 +              for (i = 0; i < pmb_nr; i++) {
 +                      struct moved_entry *p = pmb[i];
 +                      struct moved_entry *pnext = (p && p->next_line) ?
 +                                      p->next_line : NULL;
 +                      if (pnext && !hm->cmpfn(o, pnext, match, NULL)) {
 +                              pmb[i] = p->next_line;
 +                      } else {
 +                              pmb[i] = NULL;
 +                      }
 +              }
 +
 +              pmb_nr = shrink_potential_moved_blocks(pmb, pmb_nr);
 +
 +              if (pmb_nr == 0) {
 +                      /*
 +                       * The current line is the start of a new block.
 +                       * Setup the set of potential blocks.
 +                       */
 +                      for (; match; match = hashmap_get_next(hm, match)) {
 +                              ALLOC_GROW(pmb, pmb_nr + 1, pmb_alloc);
 +                              pmb[pmb_nr++] = match;
 +                      }
 +
 +                      flipped_block = (flipped_block + 1) % 2;
 +
 +                      adjust_last_block(o, n, block_length);
 +                      block_length = 0;
 +              }
 +
 +              block_length++;
 +
 +              if (flipped_block)
 +                      l->flags |= DIFF_SYMBOL_MOVED_LINE_ALT;
 +      }
 +      adjust_last_block(o, n, block_length);
 +
 +      free(pmb);
 +}
 +
 +#define DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK \
 +  (DIFF_SYMBOL_MOVED_LINE | DIFF_SYMBOL_MOVED_LINE_ALT)
 +static void dim_moved_lines(struct diff_options *o)
 +{
 +      int n;
 +      for (n = 0; n < o->emitted_symbols->nr; n++) {
 +              struct emitted_diff_symbol *prev = (n != 0) ?
 +                              &o->emitted_symbols->buf[n - 1] : NULL;
 +              struct emitted_diff_symbol *l = &o->emitted_symbols->buf[n];
 +              struct emitted_diff_symbol *next =
 +                              (n < o->emitted_symbols->nr - 1) ?
 +                              &o->emitted_symbols->buf[n + 1] : NULL;
 +
 +              /* Not a plus or minus line? */
 +              if (l->s != DIFF_SYMBOL_PLUS && l->s != DIFF_SYMBOL_MINUS)
 +                      continue;
 +
 +              /* Not a moved line? */
 +              if (!(l->flags & DIFF_SYMBOL_MOVED_LINE))
 +                      continue;
 +
 +              /*
 +               * If prev or next are not a plus or minus line,
 +               * pretend they don't exist
 +               */
 +              if (prev && prev->s != DIFF_SYMBOL_PLUS &&
 +                          prev->s != DIFF_SYMBOL_MINUS)
 +                      prev = NULL;
 +              if (next && next->s != DIFF_SYMBOL_PLUS &&
 +                          next->s != DIFF_SYMBOL_MINUS)
 +                      next = NULL;
 +
 +              /* Inside a block? */
 +              if ((prev &&
 +                  (prev->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==
 +                  (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK)) &&
 +                  (next &&
 +                  (next->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK) ==
 +                  (l->flags & DIFF_SYMBOL_MOVED_LINE_ZEBRA_MASK))) {
 +                      l->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;
 +                      continue;
 +              }
 +
 +              /* Check if we are at an interesting bound: */
 +              if (prev && (prev->flags & DIFF_SYMBOL_MOVED_LINE) &&
 +                  (prev->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=
 +                     (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))
 +                      continue;
 +              if (next && (next->flags & DIFF_SYMBOL_MOVED_LINE) &&
 +                  (next->flags & DIFF_SYMBOL_MOVED_LINE_ALT) !=
 +                     (l->flags & DIFF_SYMBOL_MOVED_LINE_ALT))
 +                      continue;
 +
 +              /*
 +               * The boundary to prev and next are not interesting,
 +               * so this line is not interesting as a whole
 +               */
 +              l->flags |= DIFF_SYMBOL_MOVED_LINE_UNINTERESTING;
 +      }
 +}
 +
 +static void emit_line_ws_markup(struct diff_options *o,
 +                              const char *set, const char *reset,
 +                              const char *line, int len, char sign,
 +                              unsigned ws_rule, int blank_at_eof)
  {
 -      const char *set = diff_get_color(ecbdata->color_diff, color);
        const char *ws = NULL;
  
 -      if (ecbdata->opt->ws_error_highlight & ws_error_highlight) {
 -              ws = diff_get_color(ecbdata->color_diff, DIFF_WHITESPACE);
 +      if (o->ws_error_highlight & ws_rule) {
 +              ws = diff_get_color_opt(o, DIFF_WHITESPACE);
                if (!*ws)
                        ws = NULL;
        }
  
        if (!ws)
 -              emit_line_0(ecbdata->opt, set, reset, sign, line, len);
 -      else if (sign == '+' && new_blank_line_at_eof(ecbdata, line, len))
 +              emit_line_0(o, set, reset, sign, line, len);
 +      else if (blank_at_eof)
                /* Blank line at EOF - paint '+' as well */
 -              emit_line_0(ecbdata->opt, ws, reset, sign, line, len);
 +              emit_line_0(o, ws, reset, sign, line, len);
        else {
                /* Emit just the prefix, then the rest. */
 -              emit_line_0(ecbdata->opt, set, reset, sign, "", 0);
 -              ws_check_emit(line, len, ecbdata->ws_rule,
 -                            ecbdata->opt->file, set, reset, ws);
 +              emit_line_0(o, set, reset, sign, "", 0);
 +              ws_check_emit(line, len, ws_rule,
 +                            o->file, set, reset, ws);
 +      }
 +}
 +
 +static void emit_diff_symbol_from_struct(struct diff_options *o,
 +                                       struct emitted_diff_symbol *eds)
 +{
 +      static const char *nneof = " No newline at end of file\n";
 +      const char *context, *reset, *set, *meta, *fraginfo;
 +      struct strbuf sb = STRBUF_INIT;
 +
 +      enum diff_symbol s = eds->s;
 +      const char *line = eds->line;
 +      int len = eds->len;
 +      unsigned flags = eds->flags;
 +
 +      switch (s) {
 +      case DIFF_SYMBOL_NO_LF_EOF:
 +              context = diff_get_color_opt(o, DIFF_CONTEXT);
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              putc('\n', o->file);
 +              emit_line_0(o, context, reset, '\\',
 +                          nneof, strlen(nneof));
 +              break;
 +      case DIFF_SYMBOL_SUBMODULE_HEADER:
 +      case DIFF_SYMBOL_SUBMODULE_ERROR:
 +      case DIFF_SYMBOL_SUBMODULE_PIPETHROUGH:
 +      case DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES:
 +      case DIFF_SYMBOL_SUMMARY:
 +      case DIFF_SYMBOL_STATS_LINE:
 +      case DIFF_SYMBOL_BINARY_DIFF_BODY:
 +      case DIFF_SYMBOL_CONTEXT_FRAGINFO:
 +              emit_line(o, "", "", line, len);
 +              break;
 +      case DIFF_SYMBOL_CONTEXT_INCOMPLETE:
 +      case DIFF_SYMBOL_CONTEXT_MARKER:
 +              context = diff_get_color_opt(o, DIFF_CONTEXT);
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              emit_line(o, context, reset, line, len);
 +              break;
 +      case DIFF_SYMBOL_SEPARATOR:
 +              fprintf(o->file, "%s%c",
 +                      diff_line_prefix(o),
 +                      o->line_termination);
 +              break;
 +      case DIFF_SYMBOL_CONTEXT:
 +              set = diff_get_color_opt(o, DIFF_CONTEXT);
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              emit_line_ws_markup(o, set, reset, line, len, ' ',
 +                                  flags & (DIFF_SYMBOL_CONTENT_WS_MASK), 0);
 +              break;
 +      case DIFF_SYMBOL_PLUS:
 +              switch (flags & (DIFF_SYMBOL_MOVED_LINE |
 +                               DIFF_SYMBOL_MOVED_LINE_ALT |
 +                               DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {
 +              case DIFF_SYMBOL_MOVED_LINE |
 +                   DIFF_SYMBOL_MOVED_LINE_ALT |
 +                   DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
 +                      set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT_DIM);
 +                      break;
 +              case DIFF_SYMBOL_MOVED_LINE |
 +                   DIFF_SYMBOL_MOVED_LINE_ALT:
 +                      set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_ALT);
 +                      break;
 +              case DIFF_SYMBOL_MOVED_LINE |
 +                   DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
 +                      set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED_DIM);
 +                      break;
 +              case DIFF_SYMBOL_MOVED_LINE:
 +                      set = diff_get_color_opt(o, DIFF_FILE_NEW_MOVED);
 +                      break;
 +              default:
 +                      set = diff_get_color_opt(o, DIFF_FILE_NEW);
 +              }
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              emit_line_ws_markup(o, set, reset, line, len, '+',
 +                                  flags & DIFF_SYMBOL_CONTENT_WS_MASK,
 +                                  flags & DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF);
 +              break;
 +      case DIFF_SYMBOL_MINUS:
 +              switch (flags & (DIFF_SYMBOL_MOVED_LINE |
 +                               DIFF_SYMBOL_MOVED_LINE_ALT |
 +                               DIFF_SYMBOL_MOVED_LINE_UNINTERESTING)) {
 +              case DIFF_SYMBOL_MOVED_LINE |
 +                   DIFF_SYMBOL_MOVED_LINE_ALT |
 +                   DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
 +                      set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT_DIM);
 +                      break;
 +              case DIFF_SYMBOL_MOVED_LINE |
 +                   DIFF_SYMBOL_MOVED_LINE_ALT:
 +                      set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_ALT);
 +                      break;
 +              case DIFF_SYMBOL_MOVED_LINE |
 +                   DIFF_SYMBOL_MOVED_LINE_UNINTERESTING:
 +                      set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED_DIM);
 +                      break;
 +              case DIFF_SYMBOL_MOVED_LINE:
 +                      set = diff_get_color_opt(o, DIFF_FILE_OLD_MOVED);
 +                      break;
 +              default:
 +                      set = diff_get_color_opt(o, DIFF_FILE_OLD);
 +              }
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              emit_line_ws_markup(o, set, reset, line, len, '-',
 +                                  flags & DIFF_SYMBOL_CONTENT_WS_MASK, 0);
 +              break;
 +      case DIFF_SYMBOL_WORDS_PORCELAIN:
 +              context = diff_get_color_opt(o, DIFF_CONTEXT);
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              emit_line(o, context, reset, line, len);
 +              fputs("~\n", o->file);
 +              break;
 +      case DIFF_SYMBOL_WORDS:
 +              context = diff_get_color_opt(o, DIFF_CONTEXT);
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              /*
 +               * Skip the prefix character, if any.  With
 +               * diff_suppress_blank_empty, there may be
 +               * none.
 +               */
 +              if (line[0] != '\n') {
 +                      line++;
 +                      len--;
 +              }
 +              emit_line(o, context, reset, line, len);
 +              break;
 +      case DIFF_SYMBOL_FILEPAIR_PLUS:
 +              meta = diff_get_color_opt(o, DIFF_METAINFO);
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              fprintf(o->file, "%s%s+++ %s%s%s\n", diff_line_prefix(o), meta,
 +                      line, reset,
 +                      strchr(line, ' ') ? "\t" : "");
 +              break;
 +      case DIFF_SYMBOL_FILEPAIR_MINUS:
 +              meta = diff_get_color_opt(o, DIFF_METAINFO);
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              fprintf(o->file, "%s%s--- %s%s%s\n", diff_line_prefix(o), meta,
 +                      line, reset,
 +                      strchr(line, ' ') ? "\t" : "");
 +              break;
 +      case DIFF_SYMBOL_BINARY_FILES:
 +      case DIFF_SYMBOL_HEADER:
 +              fprintf(o->file, "%s", line);
 +              break;
 +      case DIFF_SYMBOL_BINARY_DIFF_HEADER:
 +              fprintf(o->file, "%sGIT binary patch\n", diff_line_prefix(o));
 +              break;
 +      case DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA:
 +              fprintf(o->file, "%sdelta %s\n", diff_line_prefix(o), line);
 +              break;
 +      case DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL:
 +              fprintf(o->file, "%sliteral %s\n", diff_line_prefix(o), line);
 +              break;
 +      case DIFF_SYMBOL_BINARY_DIFF_FOOTER:
 +              fputs(diff_line_prefix(o), o->file);
 +              fputc('\n', o->file);
 +              break;
 +      case DIFF_SYMBOL_REWRITE_DIFF:
 +              fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              emit_line(o, fraginfo, reset, line, len);
 +              break;
 +      case DIFF_SYMBOL_SUBMODULE_ADD:
 +              set = diff_get_color_opt(o, DIFF_FILE_NEW);
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              emit_line(o, set, reset, line, len);
 +              break;
 +      case DIFF_SYMBOL_SUBMODULE_DEL:
 +              set = diff_get_color_opt(o, DIFF_FILE_OLD);
 +              reset = diff_get_color_opt(o, DIFF_RESET);
 +              emit_line(o, set, reset, line, len);
 +              break;
 +      case DIFF_SYMBOL_SUBMODULE_UNTRACKED:
 +              fprintf(o->file, "%sSubmodule %s contains untracked content\n",
 +                      diff_line_prefix(o), line);
 +              break;
 +      case DIFF_SYMBOL_SUBMODULE_MODIFIED:
 +              fprintf(o->file, "%sSubmodule %s contains modified content\n",
 +                      diff_line_prefix(o), line);
 +              break;
 +      case DIFF_SYMBOL_STATS_SUMMARY_NO_FILES:
 +              emit_line(o, "", "", " 0 files changed\n",
 +                        strlen(" 0 files changed\n"));
 +              break;
 +      case DIFF_SYMBOL_STATS_SUMMARY_ABBREV:
 +              emit_line(o, "", "", " ...\n", strlen(" ...\n"));
 +              break;
 +      case DIFF_SYMBOL_WORD_DIFF:
 +              fprintf(o->file, "%.*s", len, line);
 +              break;
 +      case DIFF_SYMBOL_STAT_SEP:
 +              fputs(o->stat_sep, o->file);
 +              break;
 +      default:
 +              die("BUG: unknown diff symbol");
        }
 +      strbuf_release(&sb);
 +}
 +
 +static void emit_diff_symbol(struct diff_options *o, enum diff_symbol s,
 +                           const char *line, int len, unsigned flags)
 +{
 +      struct emitted_diff_symbol e = {line, len, flags, s};
 +
 +      if (o->emitted_symbols)
 +              append_emitted_diff_symbol(o, &e);
 +      else
 +              emit_diff_symbol_from_struct(o, &e);
 +}
 +
 +void diff_emit_submodule_del(struct diff_options *o, const char *line)
 +{
 +      emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_DEL, line, strlen(line), 0);
 +}
 +
 +void diff_emit_submodule_add(struct diff_options *o, const char *line)
 +{
 +      emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ADD, line, strlen(line), 0);
 +}
 +
 +void diff_emit_submodule_untracked(struct diff_options *o, const char *path)
 +{
 +      emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_UNTRACKED,
 +                       path, strlen(path), 0);
 +}
 +
 +void diff_emit_submodule_modified(struct diff_options *o, const char *path)
 +{
 +      emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_MODIFIED,
 +                       path, strlen(path), 0);
 +}
 +
 +void diff_emit_submodule_header(struct diff_options *o, const char *header)
 +{
 +      emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_HEADER,
 +                       header, strlen(header), 0);
 +}
 +
 +void diff_emit_submodule_error(struct diff_options *o, const char *err)
 +{
 +      emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_ERROR, err, strlen(err), 0);
 +}
 +
 +void diff_emit_submodule_pipethrough(struct diff_options *o,
 +                                   const char *line, int len)
 +{
 +      emit_diff_symbol(o, DIFF_SYMBOL_SUBMODULE_PIPETHROUGH, line, len, 0);
 +}
 +
 +static int new_blank_line_at_eof(struct emit_callback *ecbdata, const char *line, int len)
 +{
 +      if (!((ecbdata->ws_rule & WS_BLANK_AT_EOF) &&
 +            ecbdata->blank_at_eof_in_preimage &&
 +            ecbdata->blank_at_eof_in_postimage &&
 +            ecbdata->blank_at_eof_in_preimage <= ecbdata->lno_in_preimage &&
 +            ecbdata->blank_at_eof_in_postimage <= ecbdata->lno_in_postimage))
 +              return 0;
 +      return ws_blank_line(line, len, ecbdata->ws_rule);
  }
  
  static void emit_add_line(const char *reset,
                          struct emit_callback *ecbdata,
                          const char *line, int len)
  {
 -      emit_line_checked(reset, ecbdata, line, len,
 -                        DIFF_FILE_NEW, WSEH_NEW, '+');
 +      unsigned flags = WSEH_NEW | ecbdata->ws_rule;
 +      if (new_blank_line_at_eof(ecbdata, line, len))
 +              flags |= DIFF_SYMBOL_CONTENT_BLANK_LINE_EOF;
 +
 +      emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_PLUS, line, len, flags);
  }
  
  static void emit_del_line(const char *reset,
                          struct emit_callback *ecbdata,
                          const char *line, int len)
  {
 -      emit_line_checked(reset, ecbdata, line, len,
 -                        DIFF_FILE_OLD, WSEH_OLD, '-');
 +      unsigned flags = WSEH_OLD | ecbdata->ws_rule;
 +      emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_MINUS, line, len, flags);
  }
  
  static void emit_context_line(const char *reset,
                              struct emit_callback *ecbdata,
                              const char *line, int len)
  {
 -      emit_line_checked(reset, ecbdata, line, len,
 -                        DIFF_CONTEXT, WSEH_CONTEXT, ' ');
 +      unsigned flags = WSEH_CONTEXT | ecbdata->ws_rule;
 +      emit_diff_symbol(ecbdata->opt, DIFF_SYMBOL_CONTEXT, line, len, flags);
  }
  
  static void emit_hunk_header(struct emit_callback *ecbdata,
        if (len < 10 ||
            memcmp(line, atat, 2) ||
            !(ep = memmem(line + 2, len - 2, atat, 2))) {
 -              emit_line(ecbdata->opt, context, reset, line, len);
 +              emit_diff_symbol(ecbdata->opt,
 +                               DIFF_SYMBOL_CONTEXT_MARKER, line, len, 0);
                return;
        }
        ep += 2; /* skip over @@ */
        }
  
        strbuf_add(&msgbuf, line + len, org_len - len);
 -      emit_line(ecbdata->opt, "", "", msgbuf.buf, msgbuf.len);
 +      strbuf_complete_line(&msgbuf);
 +      emit_diff_symbol(ecbdata->opt,
 +                       DIFF_SYMBOL_CONTEXT_FRAGINFO, msgbuf.buf, msgbuf.len, 0);
        strbuf_release(&msgbuf);
  }
  
@@@ -1416,23 -694,23 +1419,23 @@@ static void remove_tempfile(void
  {
        int i;
        for (i = 0; i < ARRAY_SIZE(diff_temp); i++) {
 -              if (is_tempfile_active(&diff_temp[i].tempfile))
 +              if (is_tempfile_active(diff_temp[i].tempfile))
                        delete_tempfile(&diff_temp[i].tempfile);
                diff_temp[i].name = NULL;
        }
  }
  
 -static void print_line_count(FILE *file, int count)
 +static void add_line_count(struct strbuf *out, int count)
  {
        switch (count) {
        case 0:
 -              fprintf(file, "0,0");
 +              strbuf_addstr(out, "0,0");
                break;
        case 1:
 -              fprintf(file, "1");
 +              strbuf_addstr(out, "1");
                break;
        default:
 -              fprintf(file, "1,%d", count);
 +              strbuf_addf(out, "1,%d", count);
                break;
        }
  }
@@@ -1441,6 -719,7 +1444,6 @@@ static void emit_rewrite_lines(struct e
                               int prefix, const char *data, int size)
  {
        const char *endp = NULL;
 -      static const char *nneof = " No newline at end of file\n";
        const char *reset = diff_get_color(ecb->color_diff, DIFF_RESET);
  
        while (0 < size) {
                size -= len;
                data += len;
        }
 -      if (!endp) {
 -              const char *context = diff_get_color(ecb->color_diff,
 -                                                   DIFF_CONTEXT);
 -              putc('\n', ecb->opt->file);
 -              emit_line_0(ecb->opt, context, reset, '\\',
 -                          nneof, strlen(nneof));
 -      }
 +      if (!endp)
 +              emit_diff_symbol(ecb->opt, DIFF_SYMBOL_NO_LF_EOF, NULL, 0, 0);
  }
  
  static void emit_rewrite_diff(const char *name_a,
                              struct diff_options *o)
  {
        int lc_a, lc_b;
 -      const char *name_a_tab, *name_b_tab;
 -      const char *metainfo = diff_get_color(o->use_color, DIFF_METAINFO);
 -      const char *fraginfo = diff_get_color(o->use_color, DIFF_FRAGINFO);
 -      const char *reset = diff_get_color(o->use_color, DIFF_RESET);
        static struct strbuf a_name = STRBUF_INIT, b_name = STRBUF_INIT;
        const char *a_prefix, *b_prefix;
        char *data_one, *data_two;
        size_t size_one, size_two;
        struct emit_callback ecbdata;
 -      const char *line_prefix = diff_line_prefix(o);
 +      struct strbuf out = STRBUF_INIT;
  
        if (diff_mnemonic_prefix && DIFF_OPT_TST(o, REVERSE_DIFF)) {
                a_prefix = o->b_prefix;
  
        name_a += (*name_a == '/');
        name_b += (*name_b == '/');
 -      name_a_tab = strchr(name_a, ' ') ? "\t" : "";
 -      name_b_tab = strchr(name_b, ' ') ? "\t" : "";
  
        strbuf_reset(&a_name);
        strbuf_reset(&b_name);
  
        lc_a = count_lines(data_one, size_one);
        lc_b = count_lines(data_two, size_two);
 -      fprintf(o->file,
 -              "%s%s--- %s%s%s\n%s%s+++ %s%s%s\n%s%s@@ -",
 -              line_prefix, metainfo, a_name.buf, name_a_tab, reset,
 -              line_prefix, metainfo, b_name.buf, name_b_tab, reset,
 -              line_prefix, fraginfo);
 +
 +      emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,
 +                       a_name.buf, a_name.len, 0);
 +      emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,
 +                       b_name.buf, b_name.len, 0);
 +
 +      strbuf_addstr(&out, "@@ -");
        if (!o->irreversible_delete)
 -              print_line_count(o->file, lc_a);
 +              add_line_count(&out, lc_a);
        else
 -              fprintf(o->file, "?,?");
 -      fprintf(o->file, " +");
 -      print_line_count(o->file, lc_b);
 -      fprintf(o->file, " @@%s\n", reset);
 +              strbuf_addstr(&out, "?,?");
 +      strbuf_addstr(&out, " +");
 +      add_line_count(&out, lc_b);
 +      strbuf_addstr(&out, " @@\n");
 +      emit_diff_symbol(o, DIFF_SYMBOL_REWRITE_DIFF, out.buf, out.len, 0);
 +      strbuf_release(&out);
 +
        if (lc_a && !o->irreversible_delete)
                emit_rewrite_lines(&ecbdata, '-', data_one, size_one);
        if (lc_b)
  
  struct diff_words_buffer {
        mmfile_t text;
 -      long alloc;
 +      unsigned long alloc;
        struct diff_words_orig {
                const char *begin, *end;
        } *orig;
@@@ -1590,49 -875,37 +1593,49 @@@ struct diff_words_data 
        struct diff_words_style *style;
  };
  
 -static int fn_out_diff_words_write_helper(FILE *fp,
 +static int fn_out_diff_words_write_helper(struct diff_options *o,
                                          struct diff_words_style_elem *st_el,
                                          const char *newline,
 -                                        size_t count, const char *buf,
 -                                        const char *line_prefix)
 +                                        size_t count, const char *buf)
  {
        int print = 0;
 +      struct strbuf sb = STRBUF_INIT;
  
        while (count) {
                char *p = memchr(buf, '\n', count);
                if (print)
 -                      fputs(line_prefix, fp);
 +                      strbuf_addstr(&sb, diff_line_prefix(o));
 +
                if (p != buf) {
 -                      if (st_el->color && fputs(st_el->color, fp) < 0)
 -                              return -1;
 -                      if (fputs(st_el->prefix, fp) < 0 ||
 -                          fwrite(buf, p ? p - buf : count, 1, fp) != 1 ||
 -                          fputs(st_el->suffix, fp) < 0)
 -                              return -1;
 -                      if (st_el->color && *st_el->color
 -                          && fputs(GIT_COLOR_RESET, fp) < 0)
 -                              return -1;
 +                      const char *reset = st_el->color && *st_el->color ?
 +                                          GIT_COLOR_RESET : NULL;
 +                      if (st_el->color && *st_el->color)
 +                              strbuf_addstr(&sb, st_el->color);
 +                      strbuf_addstr(&sb, st_el->prefix);
 +                      strbuf_add(&sb, buf, p ? p - buf : count);
 +                      strbuf_addstr(&sb, st_el->suffix);
 +                      if (reset)
 +                              strbuf_addstr(&sb, reset);
                }
                if (!p)
 -                      return 0;
 -              if (fputs(newline, fp) < 0)
 -                      return -1;
 +                      goto out;
 +
 +              strbuf_addstr(&sb, newline);
                count -= p + 1 - buf;
                buf = p + 1;
                print = 1;
 +              if (count) {
 +                      emit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,
 +                                       sb.buf, sb.len, 0);
 +                      strbuf_reset(&sb);
 +              }
        }
 +
 +out:
 +      if (sb.len)
 +              emit_diff_symbol(o, DIFF_SYMBOL_WORD_DIFF,
 +                               sb.buf, sb.len, 0);
 +      strbuf_release(&sb);
        return 0;
  }
  
@@@ -1714,20 -987,24 +1717,20 @@@ static void fn_out_diff_words_aux(void 
                fputs(line_prefix, diff_words->opt->file);
        }
        if (diff_words->current_plus != plus_begin) {
 -              fn_out_diff_words_write_helper(diff_words->opt->file,
 +              fn_out_diff_words_write_helper(diff_words->opt,
                                &style->ctx, style->newline,
                                plus_begin - diff_words->current_plus,
 -                              diff_words->current_plus, line_prefix);
 -              if (*(plus_begin - 1) == '\n')
 -                      fputs(line_prefix, diff_words->opt->file);
 +                              diff_words->current_plus);
        }
        if (minus_begin != minus_end) {
 -              fn_out_diff_words_write_helper(diff_words->opt->file,
 +              fn_out_diff_words_write_helper(diff_words->opt,
                                &style->old, style->newline,
 -                              minus_end - minus_begin, minus_begin,
 -                              line_prefix);
 +                              minus_end - minus_begin, minus_begin);
        }
        if (plus_begin != plus_end) {
 -              fn_out_diff_words_write_helper(diff_words->opt->file,
 +              fn_out_diff_words_write_helper(diff_words->opt,
                                &style->new, style->newline,
 -                              plus_end - plus_begin, plus_begin,
 -                              line_prefix);
 +                              plus_end - plus_begin, plus_begin);
        }
  
        diff_words->current_plus = plus_end;
@@@ -1821,12 -1098,11 +1824,12 @@@ static void diff_words_show(struct diff
  
        /* special case: only removal */
        if (!diff_words->plus.text.size) {
 -              fputs(line_prefix, diff_words->opt->file);
 -              fn_out_diff_words_write_helper(diff_words->opt->file,
 +              emit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,
 +                               line_prefix, strlen(line_prefix), 0);
 +              fn_out_diff_words_write_helper(diff_words->opt,
                        &style->old, style->newline,
                        diff_words->minus.text.size,
 -                      diff_words->minus.text.ptr, line_prefix);
 +                      diff_words->minus.text.ptr);
                diff_words->minus.text.size = 0;
                return;
        }
        if (diff_words->current_plus != diff_words->plus.text.ptr +
                        diff_words->plus.text.size) {
                if (color_words_output_graph_prefix(diff_words))
 -                      fputs(line_prefix, diff_words->opt->file);
 -              fn_out_diff_words_write_helper(diff_words->opt->file,
 +                      emit_diff_symbol(diff_words->opt, DIFF_SYMBOL_WORD_DIFF,
 +                                       line_prefix, strlen(line_prefix), 0);
 +              fn_out_diff_words_write_helper(diff_words->opt,
                        &style->ctx, style->newline,
                        diff_words->plus.text.ptr + diff_words->plus.text.size
 -                      - diff_words->current_plus, diff_words->current_plus,
 -                      line_prefix);
 +                      - diff_words->current_plus, diff_words->current_plus);
        }
        diff_words->minus.text.size = diff_words->plus.text.size = 0;
  }
  /* In "color-words" mode, show word-diff of words accumulated in the buffer */
  static void diff_words_flush(struct emit_callback *ecbdata)
  {
 +      struct diff_options *wo = ecbdata->diff_words->opt;
 +
        if (ecbdata->diff_words->minus.text.size ||
            ecbdata->diff_words->plus.text.size)
                diff_words_show(ecbdata->diff_words);
 +
 +      if (wo->emitted_symbols) {
 +              struct diff_options *o = ecbdata->opt;
 +              struct emitted_diff_symbols *wol = wo->emitted_symbols;
 +              int i;
 +
 +              /*
 +               * NEEDSWORK:
 +               * Instead of appending each, concat all words to a line?
 +               */
 +              for (i = 0; i < wol->nr; i++)
 +                      append_emitted_diff_symbol(o, &wol->buf[i]);
 +
 +              for (i = 0; i < wol->nr; i++)
 +                      free((void *)wol->buf[i].line);
 +
 +              wol->nr = 0;
 +      }
  }
  
  static void diff_filespec_load_driver(struct diff_filespec *one)
@@@ -1920,11 -1176,6 +1923,11 @@@ static void init_diff_words_data(struc
                xcalloc(1, sizeof(struct diff_words_data));
        ecbdata->diff_words->type = o->word_diff;
        ecbdata->diff_words->opt = o;
 +
 +      if (orig_opts->emitted_symbols)
 +              o->emitted_symbols =
 +                      xcalloc(1, sizeof(struct emitted_diff_symbols));
 +
        if (!o->word_regex)
                o->word_regex = userdiff_word_regex(one);
        if (!o->word_regex)
@@@ -1959,7 -1210,6 +1962,7 @@@ static void free_diff_words_data(struc
  {
        if (ecbdata->diff_words) {
                diff_words_flush(ecbdata);
 +              free (ecbdata->diff_words->opt->emitted_symbols);
                free (ecbdata->diff_words->opt);
                free (ecbdata->diff_words->minus.text.ptr);
                free (ecbdata->diff_words->minus.orig);
@@@ -1996,6 -1246,8 +1999,6 @@@ static unsigned long sane_truncate_line
        unsigned long allot;
        size_t l = len;
  
 -      if (ecb->truncate)
 -              return ecb->truncate(line, len);
        cp = line;
        allot = l;
        while (0 < l) {
@@@ -2024,25 -1276,30 +2027,25 @@@ static void find_lno(const char *line, 
  static void fn_out_consume(void *priv, char *line, unsigned long len)
  {
        struct emit_callback *ecbdata = priv;
 -      const char *meta = diff_get_color(ecbdata->color_diff, DIFF_METAINFO);
 -      const char *context = diff_get_color(ecbdata->color_diff, DIFF_CONTEXT);
        const char *reset = diff_get_color(ecbdata->color_diff, DIFF_RESET);
        struct diff_options *o = ecbdata->opt;
 -      const char *line_prefix = diff_line_prefix(o);
  
        o->found_changes = 1;
  
        if (ecbdata->header) {
 -              fprintf(o->file, "%s", ecbdata->header->buf);
 +              emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +                               ecbdata->header->buf, ecbdata->header->len, 0);
                strbuf_reset(ecbdata->header);
                ecbdata->header = NULL;
        }
  
        if (ecbdata->label_path[0]) {
 -              const char *name_a_tab, *name_b_tab;
 -
 -              name_a_tab = strchr(ecbdata->label_path[0], ' ') ? "\t" : "";
 -              name_b_tab = strchr(ecbdata->label_path[1], ' ') ? "\t" : "";
 -
 -              fprintf(o->file, "%s%s--- %s%s%s\n",
 -                      line_prefix, meta, ecbdata->label_path[0], reset, name_a_tab);
 -              fprintf(o->file, "%s%s+++ %s%s%s\n",
 -                      line_prefix, meta, ecbdata->label_path[1], reset, name_b_tab);
 +              emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_MINUS,
 +                               ecbdata->label_path[0],
 +                               strlen(ecbdata->label_path[0]), 0);
 +              emit_diff_symbol(o, DIFF_SYMBOL_FILEPAIR_PLUS,
 +                               ecbdata->label_path[1],
 +                               strlen(ecbdata->label_path[1]), 0);
                ecbdata->label_path[0] = ecbdata->label_path[1] = NULL;
        }
  
                len = sane_truncate_line(ecbdata, line, len);
                find_lno(line, ecbdata);
                emit_hunk_header(ecbdata, line, len);
 -              if (line[len-1] != '\n')
 -                      putc('\n', o->file);
                return;
        }
  
        if (ecbdata->diff_words) {
 +              enum diff_symbol s =
 +                      ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN ?
 +                      DIFF_SYMBOL_WORDS_PORCELAIN : DIFF_SYMBOL_WORDS;
                if (line[0] == '-') {
                        diff_words_append(line, len,
                                          &ecbdata->diff_words->minus);
                        return;
                }
                diff_words_flush(ecbdata);
 -              if (ecbdata->diff_words->type == DIFF_WORDS_PORCELAIN) {
 -                      emit_line(o, context, reset, line, len);
 -                      fputs("~\n", o->file);
 -              } else {
 -                      /*
 -                       * Skip the prefix character, if any.  With
 -                       * diff_suppress_blank_empty, there may be
 -                       * none.
 -                       */
 -                      if (line[0] != '\n') {
 -                            line++;
 -                            len--;
 -                      }
 -                      emit_line(o, context, reset, line, len);
 -              }
 +              emit_diff_symbol(o, s, line, len, 0);
                return;
        }
  
        default:
                /* incomplete line at the end */
                ecbdata->lno_in_preimage++;
 -              emit_line(o, diff_get_color(ecbdata->color_diff, DIFF_CONTEXT),
 -                        reset, line, len);
 +              emit_diff_symbol(o, DIFF_SYMBOL_CONTEXT_INCOMPLETE,
 +                               line, len, 0);
                break;
        }
  }
@@@ -2251,14 -1521,20 +2254,14 @@@ static int scale_linear(int it, int wid
        return 1 + (it * (width - 1) / max_change);
  }
  
 -static void show_name(FILE *file,
 -                    const char *prefix, const char *name, int len)
 -{
 -      fprintf(file, " %s%-*s |", prefix, len, name);
 -}
 -
 -static void show_graph(FILE *file, char ch, int cnt, const char *set, const char *reset)
 +static void show_graph(struct strbuf *out, char ch, int cnt,
 +                     const char *set, const char *reset)
  {
        if (cnt <= 0)
                return;
 -      fprintf(file, "%s", set);
 -      while (cnt--)
 -              putc(ch, file);
 -      fprintf(file, "%s", reset);
 +      strbuf_addstr(out, set);
 +      strbuf_addchars(out, ch, cnt);
 +      strbuf_addstr(out, reset);
  }
  
  static void fill_print_name(struct diffstat_file *file)
        file->print_name = pname;
  }
  
 -int print_stat_summary(FILE *fp, int files, int insertions, int deletions)
 +static void print_stat_summary_inserts_deletes(struct diff_options *options,
 +              int files, int insertions, int deletions)
  {
        struct strbuf sb = STRBUF_INIT;
 -      int ret;
  
        if (!files) {
                assert(insertions == 0 && deletions == 0);
 -              return fprintf(fp, "%s\n", " 0 files changed");
 +              emit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_NO_FILES,
 +                               NULL, 0, 0);
 +              return;
        }
  
        strbuf_addf(&sb,
                            deletions);
        }
        strbuf_addch(&sb, '\n');
 -      ret = fputs(sb.buf, fp);
 +      emit_diff_symbol(options, DIFF_SYMBOL_STATS_SUMMARY_INSERTS_DELETES,
 +                       sb.buf, sb.len, 0);
        strbuf_release(&sb);
 -      return ret;
 +}
 +
 +void print_stat_summary(FILE *fp, int files,
 +                      int insertions, int deletions)
 +{
 +      struct diff_options o;
 +      memset(&o, 0, sizeof(o));
 +      o.file = fp;
 +
 +      print_stat_summary_inserts_deletes(&o, files, insertions, deletions);
  }
  
  static void show_stats(struct diffstat_t *data, struct diff_options *options)
        int total_files = data->nr, count;
        int width, name_width, graph_width, number_width = 0, bin_width = 0;
        const char *reset, *add_c, *del_c;
 -      const char *line_prefix = "";
        int extra_shown = 0;
 +      const char *line_prefix = diff_line_prefix(options);
 +      struct strbuf out = STRBUF_INIT;
  
        if (data->nr == 0)
                return;
  
 -      line_prefix = diff_line_prefix(options);
        count = options->stat_count ? options->stat_count : data->nr;
  
        reset = diff_get_color_opt(options, DIFF_RESET);
                }
  
                if (file->is_binary) {
 -                      fprintf(options->file, "%s", line_prefix);
 -                      show_name(options->file, prefix, name, len);
 -                      fprintf(options->file, " %*s", number_width, "Bin");
 +                      strbuf_addf(&out, " %s%-*s |", prefix, len, name);
 +                      strbuf_addf(&out, " %*s", number_width, "Bin");
                        if (!added && !deleted) {
 -                              putc('\n', options->file);
 +                              strbuf_addch(&out, '\n');
 +                              emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
 +                                               out.buf, out.len, 0);
 +                              strbuf_reset(&out);
                                continue;
                        }
 -                      fprintf(options->file, " %s%"PRIuMAX"%s",
 +                      strbuf_addf(&out, " %s%"PRIuMAX"%s",
                                del_c, deleted, reset);
 -                      fprintf(options->file, " -> ");
 -                      fprintf(options->file, "%s%"PRIuMAX"%s",
 +                      strbuf_addstr(&out, " -> ");
 +                      strbuf_addf(&out, "%s%"PRIuMAX"%s",
                                add_c, added, reset);
 -                      fprintf(options->file, " bytes");
 -                      fprintf(options->file, "\n");
 +                      strbuf_addstr(&out, " bytes\n");
 +                      emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
 +                                       out.buf, out.len, 0);
 +                      strbuf_reset(&out);
                        continue;
                }
                else if (file->is_unmerged) {
 -                      fprintf(options->file, "%s", line_prefix);
 -                      show_name(options->file, prefix, name, len);
 -                      fprintf(options->file, " Unmerged\n");
 +                      strbuf_addf(&out, " %s%-*s |", prefix, len, name);
 +                      strbuf_addstr(&out, " Unmerged\n");
 +                      emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
 +                                       out.buf, out.len, 0);
 +                      strbuf_reset(&out);
                        continue;
                }
  
                                add = total - del;
                        }
                }
 -              fprintf(options->file, "%s", line_prefix);
 -              show_name(options->file, prefix, name, len);
 -              fprintf(options->file, " %*"PRIuMAX"%s",
 +              strbuf_addf(&out, " %s%-*s |", prefix, len, name);
 +              strbuf_addf(&out, " %*"PRIuMAX"%s",
                        number_width, added + deleted,
                        added + deleted ? " " : "");
 -              show_graph(options->file, '+', add, add_c, reset);
 -              show_graph(options->file, '-', del, del_c, reset);
 -              fprintf(options->file, "\n");
 +              show_graph(&out, '+', add, add_c, reset);
 +              show_graph(&out, '-', del, del_c, reset);
 +              strbuf_addch(&out, '\n');
 +              emit_diff_symbol(options, DIFF_SYMBOL_STATS_LINE,
 +                               out.buf, out.len, 0);
 +              strbuf_reset(&out);
        }
  
        for (i = 0; i < data->nr; i++) {
                if (i < count)
                        continue;
                if (!extra_shown)
 -                      fprintf(options->file, "%s ...\n", line_prefix);
 +                      emit_diff_symbol(options,
 +                                       DIFF_SYMBOL_STATS_SUMMARY_ABBREV,
 +                                       NULL, 0, 0);
                extra_shown = 1;
        }
 -      fprintf(options->file, "%s", line_prefix);
 -      print_stat_summary(options->file, total_files, adds, dels);
 +
 +      print_stat_summary_inserts_deletes(options, total_files, adds, dels);
 +      strbuf_release(&out);
  }
  
  static void show_shortstats(struct diffstat_t *data, struct diff_options *options)
  
        for (i = 0; i < data->nr; i++) {
                int added = data->files[i]->added;
 -              int deleted= data->files[i]->deleted;
 +              int deleted = data->files[i]->deleted;
  
                if (data->files[i]->is_unmerged ||
                    (!data->files[i]->is_interesting && (added + deleted == 0))) {
                        dels += deleted;
                }
        }
 -      fprintf(options->file, "%s", diff_line_prefix(options));
 -      print_stat_summary(options->file, total_files, adds, dels);
 +      print_stat_summary_inserts_deletes(options, total_files, adds, dels);
  }
  
  static void show_numstat(struct diffstat_t *data, struct diff_options *options)
@@@ -2971,8 -2225,8 +2974,8 @@@ static unsigned char *deflate_it(char *
        return deflated;
  }
  
 -static void emit_binary_diff_body(FILE *file, mmfile_t *one, mmfile_t *two,
 -                                const char *prefix)
 +static void emit_binary_diff_body(struct diff_options *o,
 +                                mmfile_t *one, mmfile_t *two)
  {
        void *cp;
        void *delta;
        }
  
        if (delta && delta_size < deflate_size) {
 -              fprintf(file, "%sdelta %lu\n", prefix, orig_size);
 +              char *s = xstrfmt("%lu", orig_size);
 +              emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_DELTA,
 +                               s, strlen(s), 0);
 +              free(s);
                free(deflated);
                data = delta;
                data_size = delta_size;
 -      }
 -      else {
 -              fprintf(file, "%sliteral %lu\n", prefix, two->size);
 +      } else {
 +              char *s = xstrfmt("%lu", two->size);
 +              emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER_LITERAL,
 +                               s, strlen(s), 0);
 +              free(s);
                free(delta);
                data = deflated;
                data_size = deflate_size;
        /* emit data encoded in base85 */
        cp = data;
        while (data_size) {
 +              int len;
                int bytes = (52 < data_size) ? 52 : data_size;
 -              char line[70];
 +              char line[71];
                data_size -= bytes;
                if (bytes <= 26)
                        line[0] = bytes + 'A' - 1;
                        line[0] = bytes - 26 + 'a' - 1;
                encode_85(line + 1, cp, bytes);
                cp = (char *) cp + bytes;
 -              fprintf(file, "%s", prefix);
 -              fputs(line, file);
 -              fputc('\n', file);
 +
 +              len = strlen(line);
 +              line[len++] = '\n';
 +              line[len] = '\0';
 +
 +              emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_BODY,
 +                               line, len, 0);
        }
 -      fprintf(file, "%s\n", prefix);
 +      emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_FOOTER, NULL, 0, 0);
        free(data);
  }
  
 -static void emit_binary_diff(FILE *file, mmfile_t *one, mmfile_t *two,
 -                           const char *prefix)
 +static void emit_binary_diff(struct diff_options *o,
 +                           mmfile_t *one, mmfile_t *two)
  {
 -      fprintf(file, "%sGIT binary patch\n", prefix);
 -      emit_binary_diff_body(file, one, two, prefix);
 -      emit_binary_diff_body(file, two, one, prefix);
 +      emit_diff_symbol(o, DIFF_SYMBOL_BINARY_DIFF_HEADER, NULL, 0, 0);
 +      emit_binary_diff_body(o, one, two);
 +      emit_binary_diff_body(o, two, one);
  }
  
  int diff_filespec_is_binary(struct diff_filespec *one)
@@@ -3125,16 -2369,24 +3128,16 @@@ static void builtin_diff(const char *na
        if (o->submodule_format == DIFF_SUBMODULE_LOG &&
            (!one->mode || S_ISGITLINK(one->mode)) &&
            (!two->mode || S_ISGITLINK(two->mode))) {
 -              const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
 -              const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
 -              show_submodule_summary(o->file, one->path ? one->path : two->path,
 -                              line_prefix,
 +              show_submodule_summary(o, one->path ? one->path : two->path,
                                &one->oid, &two->oid,
 -                              two->dirty_submodule,
 -                              meta, del, add, reset);
 +                              two->dirty_submodule);
                return;
        } else if (o->submodule_format == DIFF_SUBMODULE_INLINE_DIFF &&
                   (!one->mode || S_ISGITLINK(one->mode)) &&
                   (!two->mode || S_ISGITLINK(two->mode))) {
 -              const char *del = diff_get_color_opt(o, DIFF_FILE_OLD);
 -              const char *add = diff_get_color_opt(o, DIFF_FILE_NEW);
 -              show_submodule_inline_diff(o->file, one->path ? one->path : two->path,
 -                              line_prefix,
 +              show_submodule_inline_diff(o, one->path ? one->path : two->path,
                                &one->oid, &two->oid,
 -                              two->dirty_submodule,
 -                              meta, del, add, reset, o);
 +                              two->dirty_submodule);
                return;
        }
  
                if (complete_rewrite &&
                    (textconv_one || !diff_filespec_is_binary(one)) &&
                    (textconv_two || !diff_filespec_is_binary(two))) {
 -                      fprintf(o->file, "%s", header.buf);
 +                      emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +                                       header.buf, header.len, 0);
                        strbuf_reset(&header);
                        emit_rewrite_diff(name_a, name_b, one, two,
                                                textconv_one, textconv_two, o);
        }
  
        if (o->irreversible_delete && lbl[1][0] == '/') {
 -              fprintf(o->file, "%s", header.buf);
 +              emit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf,
 +                               header.len, 0);
                strbuf_reset(&header);
                goto free_ab_and_return;
        } else if (!DIFF_OPT_TST(o, TEXT) &&
            ( (!textconv_one && diff_filespec_is_binary(one)) ||
              (!textconv_two && diff_filespec_is_binary(two)) )) {
 +              struct strbuf sb = STRBUF_INIT;
                if (!one->data && !two->data &&
                    S_ISREG(one->mode) && S_ISREG(two->mode) &&
                    !DIFF_OPT_TST(o, BINARY)) {
                        if (!oidcmp(&one->oid, &two->oid)) {
                                if (must_show_header)
 -                                      fprintf(o->file, "%s", header.buf);
 +                                      emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +                                                       header.buf, header.len,
 +                                                       0);
                                goto free_ab_and_return;
                        }
 -                      fprintf(o->file, "%s", header.buf);
 -                      fprintf(o->file, "%sBinary files %s and %s differ\n",
 -                              line_prefix, lbl[0], lbl[1]);
 +                      emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +                                       header.buf, header.len, 0);
 +                      strbuf_addf(&sb, "%sBinary files %s and %s differ\n",
 +                                  diff_line_prefix(o), lbl[0], lbl[1]);
 +                      emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
 +                                       sb.buf, sb.len, 0);
 +                      strbuf_release(&sb);
                        goto free_ab_and_return;
                }
                if (fill_mmfile(&mf1, one) < 0 || fill_mmfile(&mf2, two) < 0)
                if (mf1.size == mf2.size &&
                    !memcmp(mf1.ptr, mf2.ptr, mf1.size)) {
                        if (must_show_header)
 -                              fprintf(o->file, "%s", header.buf);
 +                              emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +                                               header.buf, header.len, 0);
                        goto free_ab_and_return;
                }
 -              fprintf(o->file, "%s", header.buf);
 +              emit_diff_symbol(o, DIFF_SYMBOL_HEADER, header.buf, header.len, 0);
                strbuf_reset(&header);
                if (DIFF_OPT_TST(o, BINARY))
 -                      emit_binary_diff(o->file, &mf1, &mf2, line_prefix);
 -              else
 -                      fprintf(o->file, "%sBinary files %s and %s differ\n",
 -                              line_prefix, lbl[0], lbl[1]);
 +                      emit_binary_diff(o, &mf1, &mf2);
 +              else {
 +                      strbuf_addf(&sb, "%sBinary files %s and %s differ\n",
 +                                  diff_line_prefix(o), lbl[0], lbl[1]);
 +                      emit_diff_symbol(o, DIFF_SYMBOL_BINARY_FILES,
 +                                       sb.buf, sb.len, 0);
 +                      strbuf_release(&sb);
 +              }
                o->found_changes = 1;
        } else {
                /* Crazy xdl interfaces.. */
                const struct userdiff_funcname *pe;
  
                if (must_show_header) {
 -                      fprintf(o->file, "%s", header.buf);
 +                      emit_diff_symbol(o, DIFF_SYMBOL_HEADER,
 +                                       header.buf, header.len, 0);
                        strbuf_reset(&header);
                }
  
@@@ -3723,6 -2960,7 +3726,6 @@@ static void prep_temp_blob(const char *
                           const struct object_id *oid,
                           int mode)
  {
 -      int fd;
        struct strbuf buf = STRBUF_INIT;
        struct strbuf template = STRBUF_INIT;
        char *path_dup = xstrdup(path);
        strbuf_addstr(&template, "XXXXXX_");
        strbuf_addstr(&template, base);
  
 -      fd = mks_tempfile_ts(&temp->tempfile, template.buf, strlen(base) + 1);
 -      if (fd < 0)
 +      temp->tempfile = mks_tempfile_ts(template.buf, strlen(base) + 1);
 +      if (!temp->tempfile)
                die_errno("unable to create temp-file");
        if (convert_to_working_tree(path,
                        (const char *)blob, (size_t)size, &buf)) {
                blob = buf.buf;
                size = buf.len;
        }
 -      if (write_in_full(fd, blob, size) != size)
 +      if (write_in_full(temp->tempfile->fd, blob, size) < 0 ||
 +          close_tempfile_gently(temp->tempfile))
                die_errno("unable to write temp-file");
 -      close_tempfile(&temp->tempfile);
 -      temp->name = get_tempfile_path(&temp->tempfile);
 +      temp->name = get_tempfile_path(temp->tempfile);
        oid_to_hex_r(temp->hex, oid);
        xsnprintf(temp->mode, sizeof(temp->mode), "%06o", mode);
        strbuf_release(&buf);
@@@ -4011,7 -3249,7 +4014,7 @@@ static void diff_fill_oid_info(struct d
                        }
                        if (lstat(one->path, &st) < 0)
                                die_errno("stat '%s'", one->path);
 -                      if (index_path(one->oid.hash, one->path, &st, 0))
 +                      if (index_path(&one->oid, one->path, &st, 0))
                                die("cannot hash %s", one->path);
                }
        }
@@@ -4044,8 -3282,8 +4047,8 @@@ static void run_diff(struct diff_filepa
        const char *other;
        const char *attr_path;
  
 -      name  = p->one->path;
 -      other = (strcmp(name, p->two->path) ? p->two->path : NULL);
 +      name  = one->path;
 +      other = (strcmp(name, two->path) ? two->path : NULL);
        attr_path = name;
        if (o->prefix_length)
                strip_prefix(o->prefix_length, &name, &other);
@@@ -4168,8 -3406,6 +4171,8 @@@ void diff_setup(struct diff_options *op
                options->a_prefix = "a/";
                options->b_prefix = "b/";
        }
 +
 +      options->color_moved = diff_color_moved_default;
  }
  
  void diff_setup_done(struct diff_options *options)
  
        if (DIFF_OPT_TST(options, FOLLOW_RENAMES) && options->pathspec.nr != 1)
                die(_("--follow requires exactly one pathspec"));
 +
 +      if (!options->use_color || external_diff())
 +              options->color_moved = 0;
  }
  
  static int opt_arg(const char *arg, int arg_short, const char *arg_long, int *val)
@@@ -4706,19 -3939,7 +4709,19 @@@ int diff_opt_parse(struct diff_options 
        }
        else if (!strcmp(arg, "--no-color"))
                options->use_color = 0;
 -      else if (!strcmp(arg, "--color-words")) {
 +      else if (!strcmp(arg, "--color-moved")) {
 +              if (diff_color_moved_default)
 +                      options->color_moved = diff_color_moved_default;
 +              if (options->color_moved == COLOR_MOVED_NO)
 +                      options->color_moved = COLOR_MOVED_DEFAULT;
 +      } else if (!strcmp(arg, "--no-color-moved"))
 +              options->color_moved = COLOR_MOVED_NO;
 +      else if (skip_prefix(arg, "--color-moved=", &arg)) {
 +              int cm = parse_color_moved(arg);
 +              if (cm < 0)
 +                      die("bad --color-moved argument: %s", arg);
 +              options->color_moved = cm;
 +      } else if (!strcmp(arg, "--color-words")) {
                options->use_color = 1;
                options->word_diff = DIFF_WORDS_COLOR;
        }
@@@ -5248,79 -4469,67 +5251,79 @@@ static void flush_one_pair(struct diff_
        }
  }
  
 -static void show_file_mode_name(FILE *file, const char *newdelete, struct diff_filespec *fs)
 +static void show_file_mode_name(struct diff_options *opt, const char *newdelete, struct diff_filespec *fs)
  {
 +      struct strbuf sb = STRBUF_INIT;
        if (fs->mode)
 -              fprintf(file, " %s mode %06o ", newdelete, fs->mode);
 +              strbuf_addf(&sb, " %s mode %06o ", newdelete, fs->mode);
        else
 -              fprintf(file, " %s ", newdelete);
 -      write_name_quoted(fs->path, file, '\n');
 -}
 +              strbuf_addf(&sb, " %s ", newdelete);
  
 +      quote_c_style(fs->path, &sb, NULL, 0);
 +      strbuf_addch(&sb, '\n');
 +      emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
 +                       sb.buf, sb.len, 0);
 +      strbuf_release(&sb);
 +}
  
 -static void show_mode_change(FILE *file, struct diff_filepair *p, int show_name,
 -              const char *line_prefix)
 +static void show_mode_change(struct diff_options *opt, struct diff_filepair *p,
 +              int show_name)
  {
        if (p->one->mode && p->two->mode && p->one->mode != p->two->mode) {
 -              fprintf(file, "%s mode change %06o => %06o%c", line_prefix, p->one->mode,
 -                      p->two->mode, show_name ? ' ' : '\n');
 +              struct strbuf sb = STRBUF_INIT;
 +              strbuf_addf(&sb, " mode change %06o => %06o",
 +                          p->one->mode, p->two->mode);
                if (show_name) {
 -                      write_name_quoted(p->two->path, file, '\n');
 +                      strbuf_addch(&sb, ' ');
 +                      quote_c_style(p->two->path, &sb, NULL, 0);
                }
 +              strbuf_addch(&sb, '\n');
 +              emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
 +                               sb.buf, sb.len, 0);
 +              strbuf_release(&sb);
        }
  }
  
 -static void show_rename_copy(FILE *file, const char *renamecopy, struct diff_filepair *p,
 -                      const char *line_prefix)
 +static void show_rename_copy(struct diff_options *opt, const char *renamecopy,
 +              struct diff_filepair *p)
  {
 +      struct strbuf sb = STRBUF_INIT;
        char *names = pprint_rename(p->one->path, p->two->path);
 -
 -      fprintf(file, " %s %s (%d%%)\n", renamecopy, names, similarity_index(p));
 +      strbuf_addf(&sb, " %s %s (%d%%)\n",
 +                      renamecopy, names, similarity_index(p));
        free(names);
 -      show_mode_change(file, p, 0, line_prefix);
 +      emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
 +                               sb.buf, sb.len, 0);
 +      show_mode_change(opt, p, 0);
 +      strbuf_release(&sb);
  }
  
  static void diff_summary(struct diff_options *opt, struct diff_filepair *p)
  {
 -      FILE *file = opt->file;
 -      const char *line_prefix = diff_line_prefix(opt);
 -
        switch(p->status) {
        case DIFF_STATUS_DELETED:
 -              fputs(line_prefix, file);
 -              show_file_mode_name(file, "delete", p->one);
 +              show_file_mode_name(opt, "delete", p->one);
                break;
        case DIFF_STATUS_ADDED:
 -              fputs(line_prefix, file);
 -              show_file_mode_name(file, "create", p->two);
 +              show_file_mode_name(opt, "create", p->two);
                break;
        case DIFF_STATUS_COPIED:
 -              fputs(line_prefix, file);
 -              show_rename_copy(file, "copy", p, line_prefix);
 +              show_rename_copy(opt, "copy", p);
                break;
        case DIFF_STATUS_RENAMED:
 -              fputs(line_prefix, file);
 -              show_rename_copy(file, "rename", p, line_prefix);
 +              show_rename_copy(opt, "rename", p);
                break;
        default:
                if (p->score) {
 -                      fprintf(file, "%s rewrite ", line_prefix);
 -                      write_name_quoted(p->two->path, file, ' ');
 -                      fprintf(file, "(%d%%)\n", similarity_index(p));
 +                      struct strbuf sb = STRBUF_INIT;
 +                      strbuf_addstr(&sb, " rewrite ");
 +                      quote_c_style(p->two->path, &sb, NULL, 0);
 +                      strbuf_addf(&sb, " (%d%%)\n", similarity_index(p));
 +                      emit_diff_symbol(opt, DIFF_SYMBOL_SUMMARY,
 +                                       sb.buf, sb.len, 0);
 +                      strbuf_release(&sb);
                }
 -              show_mode_change(file, p, !p->score, line_prefix);
 +              show_mode_change(opt, p, !p->score);
                break;
        }
  }
@@@ -5525,51 -4734,6 +5528,51 @@@ void diff_warn_rename_limit(const char 
                warning(_(rename_limit_advice), varname, needed);
  }
  
 +static void diff_flush_patch_all_file_pairs(struct diff_options *o)
 +{
 +      int i;
 +      static struct emitted_diff_symbols esm = EMITTED_DIFF_SYMBOLS_INIT;
 +      struct diff_queue_struct *q = &diff_queued_diff;
 +
 +      if (WSEH_NEW & WS_RULE_MASK)
 +              die("BUG: WS rules bit mask overlaps with diff symbol flags");
 +
 +      if (o->color_moved)
 +              o->emitted_symbols = &esm;
 +
 +      for (i = 0; i < q->nr; i++) {
 +              struct diff_filepair *p = q->queue[i];
 +              if (check_pair_status(p))
 +                      diff_flush_patch(p, o);
 +      }
 +
 +      if (o->emitted_symbols) {
 +              if (o->color_moved) {
 +                      struct hashmap add_lines, del_lines;
 +
 +                      hashmap_init(&del_lines,
 +                                   (hashmap_cmp_fn)moved_entry_cmp, o, 0);
 +                      hashmap_init(&add_lines,
 +                                   (hashmap_cmp_fn)moved_entry_cmp, o, 0);
 +
 +                      add_lines_to_move_detection(o, &add_lines, &del_lines);
 +                      mark_color_as_moved(o, &add_lines, &del_lines);
 +                      if (o->color_moved == COLOR_MOVED_ZEBRA_DIM)
 +                              dim_moved_lines(o);
 +
 +                      hashmap_free(&add_lines, 0);
 +                      hashmap_free(&del_lines, 0);
 +              }
 +
 +              for (i = 0; i < esm.nr; i++)
 +                      emit_diff_symbol_from_struct(o, &esm.buf[i]);
 +
 +              for (i = 0; i < esm.nr; i++)
 +                      free((void *)esm.buf[i].line);
 +      }
 +      esm.nr = 0;
 +}
 +
  void diff_flush(struct diff_options *options)
  {
        struct diff_queue_struct *q = &diff_queued_diff;
                        fclose(options->file);
                options->file = xfopen("/dev/null", "w");
                options->close_file = 1;
 +              options->color_moved = 0;
                for (i = 0; i < q->nr; i++) {
                        struct diff_filepair *p = q->queue[i];
                        if (check_pair_status(p))
  
        if (output_format & DIFF_FORMAT_PATCH) {
                if (separator) {
 -                      fprintf(options->file, "%s%c",
 -                              diff_line_prefix(options),
 -                              options->line_termination);
 -                      if (options->stat_sep) {
 +                      emit_diff_symbol(options, DIFF_SYMBOL_SEPARATOR, NULL, 0, 0);
 +                      if (options->stat_sep)
                                /* attach patch instead of inline */
 -                              fputs(options->stat_sep, options->file);
 -                      }
 +                              emit_diff_symbol(options, DIFF_SYMBOL_STAT_SEP,
 +                                               NULL, 0, 0);
                }
  
 -              for (i = 0; i < q->nr; i++) {
 -                      struct diff_filepair *p = q->queue[i];
 -                      if (check_pair_status(p))
 -                              diff_flush_patch(p, options);
 -              }
 +              diff_flush_patch_all_file_pairs(options);
        }
  
        if (output_format & DIFF_FORMAT_CALLBACK)
diff --combined t/t6300-for-each-ref.sh
index 416ff7d0b832d2898173dd2ac83201cf8e02955b,09f2b7799d5f6126d3b52cee17e52f9cf5a0b1e3..3aa534933e08f85ea1c215e5ef6cc6c7f4a7cd31
@@@ -51,7 -51,6 +51,7 @@@ test_atom() 
  }
  
  test_atom head refname refs/heads/master
 +test_atom head refname: refs/heads/master
  test_atom head refname:short master
  test_atom head refname:lstrip=1 heads/master
  test_atom head refname:lstrip=2 master
@@@ -442,6 -441,11 +442,11 @@@ test_expect_success '--color can overri
        test_cmp expected.color actual
  '
  
+ test_expect_success 'color.ui=always does not override tty check' '
+       git -c color.ui=always for-each-ref --format="$color_format" >actual &&
+       test_cmp expected.bare actual
+ '
  cat >expected <<\EOF
  heads/master
  tags/master
@@@ -605,104 -609,18 +610,104 @@@ test_expect_success 'do not dereferenc
  cat >trailers <<EOF
  Reviewed-by: A U Thor <author@example.com>
  Signed-off-by: A U Thor <author@example.com>
 +[ v2 updated patch description ]
 +Acked-by: A U Thor
 +  <author@example.com>
  EOF
  
 -test_expect_success 'basic atom: head contents:trailers' '
 +unfold () {
 +      perl -0pe 's/\n\s+/ /g'
 +}
 +
 +test_expect_success 'set up trailers for next test' '
        echo "Some contents" > two &&
        git add two &&
 -      git commit -F - <<-EOF &&
 +      git commit -F - <<-EOF
        trailers: this commit message has trailers
  
        Some message contents
  
        $(cat trailers)
        EOF
 +'
 +
 +test_expect_success '%(trailers:unfold) unfolds trailers' '
 +      git for-each-ref --format="%(trailers:unfold)" refs/heads/master >actual &&
 +      {
 +              unfold <trailers
 +              echo
 +      } >expect &&
 +      test_cmp expect actual
 +'
 +
 +test_expect_success '%(trailers:only) shows only "key: value" trailers' '
 +      git for-each-ref --format="%(trailers:only)" refs/heads/master >actual &&
 +      {
 +              grep -v patch.description <trailers &&
 +              echo
 +      } >expect &&
 +      test_cmp expect actual
 +'
 +
 +test_expect_success '%(trailers:only) and %(trailers:unfold) work together' '
 +      git for-each-ref --format="%(trailers:only,unfold)" refs/heads/master >actual &&
 +      git for-each-ref --format="%(trailers:unfold,only)" refs/heads/master >reverse &&
 +      test_cmp actual reverse &&
 +      {
 +              grep -v patch.description <trailers | unfold &&
 +              echo
 +      } >expect &&
 +      test_cmp expect actual
 +'
 +
 +test_expect_success '%(contents:trailers:unfold) unfolds trailers' '
 +      git for-each-ref --format="%(contents:trailers:unfold)" refs/heads/master >actual &&
 +      {
 +              unfold <trailers
 +              echo
 +      } >expect &&
 +      test_cmp expect actual
 +'
 +
 +test_expect_success '%(contents:trailers:only) shows only "key: value" trailers' '
 +      git for-each-ref --format="%(contents:trailers:only)" refs/heads/master >actual &&
 +      {
 +              grep -v patch.description <trailers &&
 +              echo
 +      } >expect &&
 +      test_cmp expect actual
 +'
 +
 +test_expect_success '%(contents:trailers:only) and %(contents:trailers:unfold) work together' '
 +      git for-each-ref --format="%(contents:trailers:only,unfold)" refs/heads/master >actual &&
 +      git for-each-ref --format="%(contents:trailers:unfold,only)" refs/heads/master >reverse &&
 +      test_cmp actual reverse &&
 +      {
 +              grep -v patch.description <trailers | unfold &&
 +              echo
 +      } >expect &&
 +      test_cmp expect actual
 +'
 +
 +test_expect_success '%(trailers) rejects unknown trailers arguments' '
 +      # error message cannot be checked under i18n
 +      cat >expect <<-EOF &&
 +      fatal: unknown %(trailers) argument: unsupported
 +      EOF
 +      test_must_fail git for-each-ref --format="%(trailers:unsupported)" 2>actual &&
 +      test_i18ncmp expect actual
 +'
 +
 +test_expect_success '%(contents:trailers) rejects unknown trailers arguments' '
 +      # error message cannot be checked under i18n
 +      cat >expect <<-EOF &&
 +      fatal: unknown %(trailers) argument: unsupported
 +      EOF
 +      test_must_fail git for-each-ref --format="%(contents:trailers:unsupported)" 2>actual &&
 +      test_i18ncmp expect actual
 +'
 +
 +test_expect_success 'basic atom: head contents:trailers' '
        git for-each-ref --format="%(contents:trailers)" refs/heads/master >actual &&
        sanitize_pgp <actual >actual.clean &&
        # git for-each-ref ends with a blank line
diff --combined t/t7004-tag.sh
index 4e62c505fc9ce84adf4b0f74110b39d18c571670,62aa322846be36a2794d4089369292cecd319d6b..a9af2de9960b345878ac0f85c33b1efd3e038d28
@@@ -1863,6 -1863,13 +1863,6 @@@ test_expect_success 'version sort with 
        git tag -l --sort=version:refname
  '
  
 -run_with_limited_stack () {
 -      (ulimit -s 128 && "$@")
 -}
 -
 -test_lazy_prereq ULIMIT_STACK_SIZE 'run_with_limited_stack true'
 -
 -# we require ulimit, this excludes Windows
  test_expect_success ULIMIT_STACK_SIZE '--contains and --no-contains work in a deep repo' '
        >expect &&
        i=1 &&
@@@ -1918,6 -1925,12 +1918,12 @@@ test_expect_success '--color overrides 
        test_cmp expect.color actual
  '
  
+ test_expect_success 'color.ui=always overrides auto-color' '
+       git -c color.ui=always tag $color_args >actual.raw &&
+       test_decode_color <actual.raw >actual &&
+       test_cmp expect.color actual
+ '
  test_expect_success 'setup --merged test tags' '
        git tag mergetest-1 HEAD~2 &&
        git tag mergetest-2 HEAD~1 &&