From: Junio C Hamano <gitster@pobox.com>
Date: Thu, 16 Dec 2010 20:58:38 +0000 (-0800)
Subject: Merge branch 'jn/fast-import-blob-access'
X-Git-Tag: v1.7.4-rc0~30
X-Git-Url: https://git.lorimer.id.au/gitweb.git/diff_plain/914584266c7675d3d0fa18e0eed05742327b22e5?ds=inline;hp=-c

Merge branch 'jn/fast-import-blob-access'

* jn/fast-import-blob-access:
t9300: avoid short reads from dd
t9300: remove unnecessary use of /dev/stdin
fast-import: Allow cat-blob requests at arbitrary points in stream
fast-import: let importers retrieve blobs
fast-import: clarify documentation of "feature" command
fast-import: stricter parsing of integer options

Conflicts:
fast-import.c
---

914584266c7675d3d0fa18e0eed05742327b22e5
diff --combined Documentation/git-fast-import.txt
index 9667e9aebc,534b2519b3..f56dfcabb9
--- a/Documentation/git-fast-import.txt
+++ b/Documentation/git-fast-import.txt
@@@ -92,6 -92,11 +92,11 @@@ OPTION
  	--(no-)-relative-marks= with the --(import|export)-marks=
  	options.
  
+ --cat-blob-fd=<fd>::
+ 	Specify the file descriptor that will be written to
+ 	when the `cat-blob` command is encountered in the stream.
+ 	The default behaviour is to write to `stdout`.
+ 
  --export-pack-edges=<file>::
  	After creating a packfile, print a line of data to
  	<file> listing the filename of the packfile and the last
@@@ -320,6 -325,11 +325,11 @@@ and control the current import process
  	standard output.  This command is optional and is not needed
  	to perform an import.
  
+ `cat-blob`::
+ 	Causes fast-import to print a blob in 'cat-file --batch'
+ 	format to the file descriptor set with `--cat-blob-fd` or
+ 	`stdout` if unspecified.
+ 
  `feature`::
  	Require that fast-import supports the specified feature, or
  	abort if it does not.
@@@ -879,34 -889,65 +889,65 @@@ Placing a `progress` command immediatel
  inform the reader when the `checkpoint` has been completed and it
  can safely access the refs that fast-import updated.
  
+ `cat-blob`
+ ~~~~~~~~~~
+ Causes fast-import to print a blob to a file descriptor previously
+ arranged with the `--cat-blob-fd` argument.  The command otherwise
+ has no impact on the current import; its main purpose is to
+ retrieve blobs that may be in fast-import's memory but not
+ accessible from the target repository.
+ 
+ ....
+ 	'cat-blob' SP <dataref> LF
+ ....
+ 
+ The `<dataref>` can be either a mark reference (`:<idnum>`)
+ set previously or a full 40-byte SHA-1 of a Git blob, preexisting or
+ ready to be written.
+ 
+ output uses the same format as `git cat-file --batch`:
+ 
+ ====
+ 	<sha1> SP 'blob' SP <size> LF
+ 	<contents> LF
+ ====
+ 
+ This command can be used anywhere in the stream that comments are
+ accepted.  In particular, the `cat-blob` command can be used in the
+ middle of a commit but not in the middle of a `data` command.
+ 
  `feature`
  ~~~~~~~~~
  Require that fast-import supports the specified feature, or abort if
  it does not.
  
  ....
- 	'feature' SP <feature> LF
+ 	'feature' SP <feature> ('=' <argument>)? LF
  ....
  
- The <feature> part of the command may be any string matching
- ^[a-zA-Z][a-zA-Z-]*$ and should be understood by fast-import.
- 
- Feature work identical as their option counterparts with the
- exception of the import-marks feature, see below.
+ The <feature> part of the command may be any one of the following:
  
- The following features are currently supported:
+ date-format::
+ export-marks::
+ relative-marks::
+ no-relative-marks::
+ force::
+ 	Act as though the corresponding command-line option with
+ 	a leading '--' was passed on the command line
+ 	(see OPTIONS, above).
  
- * date-format
- * import-marks
- * export-marks
- * relative-marks
- * no-relative-marks
- * force
+ import-marks::
+ 	Like --import-marks except in two respects: first, only one
+ 	"feature import-marks" command is allowed per stream;
+ 	second, an --import-marks= command-line option overrides
+ 	any "feature import-marks" command in the stream.
  
- The import-marks behaves differently from when it is specified as
- commandline option in that only one "feature import-marks" is allowed
- per stream. Also, any --import-marks= specified on the commandline
- will override those from the stream (if any).
+ cat-blob::
+ 	Ignored.  Versions of fast-import not supporting the
+ 	"cat-blob" command will exit with a message indicating so.
+ 	This lets the import error out early with a clear message,
+ 	rather than wasting time on the early part of an import
+ 	before the unsupported command is detected.
  
  `option`
  ~~~~~~~~
@@@ -933,6 -974,7 +974,7 @@@ not be passed as option
  * date-format
  * import-marks
  * export-marks
+ * cat-blob-fd
  * force
  
  Crash Reports
@@@ -1233,13 -1275,6 +1275,13 @@@ and lazy loading of subtrees, allows fa
  projects with 2,000+ branches and 45,114+ files in a very limited
  memory footprint (less than 2.7 MiB per active branch).
  
 +Signals
 +-------
 +Sending *SIGUSR1* to the 'git fast-import' process ends the current
 +packfile early, simulating a `checkpoint` command.  The impatient
 +operator can use this facility to peek at the objects and refs from an
 +import in progress, at the cost of some added running time and worse
 +compression.
  
  Author
  ------
diff --combined fast-import.c
index 3c58e6f04a,f71ea3e7bc..785776086c
--- a/fast-import.c
+++ b/fast-import.c
@@@ -132,14 -132,17 +132,17 @@@ Format of STDIN stream
    ts    ::= # time since the epoch in seconds, ascii base10 notation;
    tz    ::= # GIT style timezone;
  
-      # note: comments may appear anywhere in the input, except
-      # within a data command.  Any form of the data command
-      # always escapes the related input from comment processing.
+      # note: comments and cat requests may appear anywhere
+      # in the input, except within a data command.  Any form
+      # of the data command always escapes the related input
+      # from comment processing.
       #
       # In case it is not clear, the '#' that starts the comment
       # must be the first character on that line (an lf
       # preceded it).
       #
+   cat_blob ::= 'cat-blob' sp (hexsha1 | idnum) lf;
+ 
    comment ::= '#' not_lf* lf;
    not_lf  ::= # Any byte that is not ASCII newline (LF);
  */
@@@ -156,7 -159,6 +159,7 @@@
  #include "csum-file.h"
  #include "quote.h"
  #include "exec_cmd.h"
 +#include "dir.h"
  
  #define PACK_ID_BITS 16
  #define MAX_PACK_ID ((1<<PACK_ID_BITS)-1)
@@@ -362,10 -364,11 +365,14 @@@ static uintmax_t next_mark
  static struct strbuf new_data = STRBUF_INIT;
  static int seen_data_command;
  
 +/* Signal handling */
 +static volatile sig_atomic_t checkpoint_requested;
 +
+ /* Where to write output of cat-blob commands */
+ static int cat_blob_fd = STDOUT_FILENO;
+ 
  static void parse_argv(void);
+ static void parse_cat_blob(void);
  
  static void write_branch_report(FILE *rpt, struct branch *b)
  {
@@@ -504,32 -507,6 +511,32 @@@ static NORETURN void die_nicely(const c
  	exit(128);
  }
  
 +#ifndef SIGUSR1	/* Windows, for example */
 +
 +static void set_checkpoint_signal(void)
 +{
 +}
 +
 +#else
 +
 +static void checkpoint_signal(int signo)
 +{
 +	checkpoint_requested = 1;
 +}
 +
 +static void set_checkpoint_signal(void)
 +{
 +	struct sigaction sa;
 +
 +	memset(&sa, 0, sizeof(sa));
 +	sa.sa_handler = checkpoint_signal;
 +	sigemptyset(&sa.sa_mask);
 +	sa.sa_flags = SA_RESTART;
 +	sigaction(SIGUSR1, &sa, NULL);
 +}
 +
 +#endif
 +
  static void alloc_objects(unsigned int cnt)
  {
  	struct object_entry_pool *b;
@@@ -569,17 -546,22 +576,17 @@@ static struct object_entry *insert_obje
  {
  	unsigned int h = sha1[0] << 8 | sha1[1];
  	struct object_entry *e = object_table[h];
 -	struct object_entry *p = NULL;
  
  	while (e) {
  		if (!hashcmp(sha1, e->idx.sha1))
  			return e;
 -		p = e;
  		e = e->next;
  	}
  
  	e = new_object(sha1);
 -	e->next = NULL;
 +	e->next = object_table[h];
  	e->idx.offset = 0;
 -	if (p)
 -		p->next = e;
 -	else
 -		object_table[h] = e;
 +	object_table[h] = e;
  	return e;
  }
  
@@@ -1462,20 -1444,6 +1469,20 @@@ static void store_tree(struct tree_entr
  	t->entry_count -= del;
  }
  
 +static void tree_content_replace(
 +	struct tree_entry *root,
 +	const unsigned char *sha1,
 +	const uint16_t mode,
 +	struct tree_content *newtree)
 +{
 +	if (!S_ISDIR(mode))
 +		die("Root cannot be a non-directory");
 +	hashcpy(root->versions[1].sha1, sha1);
 +	if (root->tree)
 +		release_tree_content_recursive(root->tree);
 +	root->tree = newtree;
 +}
 +
  static int tree_content_set(
  	struct tree_entry *root,
  	const char *p,
@@@ -1483,7 -1451,7 +1490,7 @@@
  	const uint16_t mode,
  	struct tree_content *subtree)
  {
 -	struct tree_content *t = root->tree;
 +	struct tree_content *t;
  	const char *slash1;
  	unsigned int i, n;
  	struct tree_entry *e;
@@@ -1493,17 -1461,23 +1500,17 @@@
  		n = slash1 - p;
  	else
  		n = strlen(p);
 -	if (!slash1 && !n) {
 -		if (!S_ISDIR(mode))
 -			die("Root cannot be a non-directory");
 -		hashcpy(root->versions[1].sha1, sha1);
 -		if (root->tree)
 -			release_tree_content_recursive(root->tree);
 -		root->tree = subtree;
 -		return 1;
 -	}
  	if (!n)
  		die("Empty path component found in input");
  	if (!slash1 && !S_ISDIR(mode) && subtree)
  		die("Non-directories cannot have subtrees");
  
 +	if (!root->tree)
 +		load_tree(root);
 +	t = root->tree;
  	for (i = 0; i < t->entry_count; i++) {
  		e = t->entries[i];
 -		if (e->name->str_len == n && !strncmp(p, e->name->str_dat, n)) {
 +		if (e->name->str_len == n && !strncmp_icase(p, e->name->str_dat, n)) {
  			if (!slash1) {
  				if (!S_ISDIR(mode)
  						&& e->versions[1].mode == mode
@@@ -1556,7 -1530,7 +1563,7 @@@ static int tree_content_remove
  	const char *p,
  	struct tree_entry *backup_leaf)
  {
 -	struct tree_content *t = root->tree;
 +	struct tree_content *t;
  	const char *slash1;
  	unsigned int i, n;
  	struct tree_entry *e;
@@@ -1567,12 -1541,9 +1574,12 @@@
  	else
  		n = strlen(p);
  
 +	if (!root->tree)
 +		load_tree(root);
 +	t = root->tree;
  	for (i = 0; i < t->entry_count; i++) {
  		e = t->entries[i];
 -		if (e->name->str_len == n && !strncmp(p, e->name->str_dat, n)) {
 +		if (e->name->str_len == n && !strncmp_icase(p, e->name->str_dat, n)) {
  			if (slash1 && !S_ISDIR(e->versions[1].mode))
  				/*
  				 * If p names a file in some subdirectory, and a
@@@ -1617,7 -1588,7 +1624,7 @@@ static int tree_content_get
  	const char *p,
  	struct tree_entry *leaf)
  {
 -	struct tree_content *t = root->tree;
 +	struct tree_content *t;
  	const char *slash1;
  	unsigned int i, n;
  	struct tree_entry *e;
@@@ -1628,12 -1599,9 +1635,12 @@@
  	else
  		n = strlen(p);
  
 +	if (!root->tree)
 +		load_tree(root);
 +	t = root->tree;
  	for (i = 0; i < t->entry_count; i++) {
  		e = t->entries[i];
 -		if (e->name->str_len == n && !strncmp(p, e->name->str_dat, n)) {
 +		if (e->name->str_len == n && !strncmp_icase(p, e->name->str_dat, n)) {
  			if (!slash1) {
  				memcpy(leaf, e, sizeof(*leaf));
  				if (e->tree && is_null_sha1(e->versions[1].sha1))
@@@ -1819,7 -1787,6 +1826,7 @@@ static void read_marks(void
  	fclose(f);
  }
  
 +
  static int read_next_command(void)
  {
  	static int stdin_eof = 0;
@@@ -1829,7 -1796,7 +1836,7 @@@
  		return EOF;
  	}
  
- 	do {
+ 	for (;;) {
  		if (unread_command_buf) {
  			unread_command_buf = 0;
  		} else {
@@@ -1862,9 -1829,14 +1869,14 @@@
  			rc->prev->next = rc;
  			cmd_tail = rc;
  		}
- 	} while (command_buf.buf[0] == '#');
- 
- 	return 0;
+ 		if (!prefixcmp(command_buf.buf, "cat-blob ")) {
+ 			parse_cat_blob();
+ 			continue;
+ 		}
+ 		if (command_buf.buf[0] == '#')
+ 			continue;
+ 		return 0;
+ 	}
  }
  
  static void skip_optional_lf(void)
@@@ -2257,10 -2229,6 +2269,10 @@@ static void file_change_m(struct branc
  				command_buf.buf);
  	}
  
 +	if (!*p) {
 +		tree_content_replace(&b->branch_tree, sha1, mode, NULL);
 +		return;
 +	}
  	tree_content_set(&b->branch_tree, p, sha1, mode, NULL);
  }
  
@@@ -2319,13 -2287,6 +2331,13 @@@ static void file_change_cr(struct branc
  		tree_content_get(&b->branch_tree, s, &leaf);
  	if (!leaf.versions[1].mode)
  		die("Path %s not in branch", s);
 +	if (!*d) {	/* C "path/to/subdir" "" */
 +		tree_content_replace(&b->branch_tree,
 +			leaf.versions[1].sha1,
 +			leaf.versions[1].mode,
 +			leaf.tree);
 +		return;
 +	}
  	tree_content_set(&b->branch_tree, d,
  		leaf.versions[1].sha1,
  		leaf.versions[1].mode,
@@@ -2739,20 -2700,89 +2751,95 @@@ static void parse_reset_branch(void
  		unread_command_buf = 1;
  }
  
+ static void cat_blob_write(const char *buf, unsigned long size)
+ {
+ 	if (write_in_full(cat_blob_fd, buf, size) != size)
+ 		die_errno("Write to frontend failed");
+ }
+ 
+ static void cat_blob(struct object_entry *oe, unsigned char sha1[20])
+ {
+ 	struct strbuf line = STRBUF_INIT;
+ 	unsigned long size;
+ 	enum object_type type = 0;
+ 	char *buf;
+ 
+ 	if (!oe || oe->pack_id == MAX_PACK_ID) {
+ 		buf = read_sha1_file(sha1, &type, &size);
+ 	} else {
+ 		type = oe->type;
+ 		buf = gfi_unpack_entry(oe, &size);
+ 	}
+ 
+ 	/*
+ 	 * Output based on batch_one_object() from cat-file.c.
+ 	 */
+ 	if (type <= 0) {
+ 		strbuf_reset(&line);
+ 		strbuf_addf(&line, "%s missing\n", sha1_to_hex(sha1));
+ 		cat_blob_write(line.buf, line.len);
+ 		strbuf_release(&line);
+ 		free(buf);
+ 		return;
+ 	}
+ 	if (!buf)
+ 		die("Can't read object %s", sha1_to_hex(sha1));
+ 	if (type != OBJ_BLOB)
+ 		die("Object %s is a %s but a blob was expected.",
+ 		    sha1_to_hex(sha1), typename(type));
+ 	strbuf_reset(&line);
+ 	strbuf_addf(&line, "%s %s %lu\n", sha1_to_hex(sha1),
+ 						typename(type), size);
+ 	cat_blob_write(line.buf, line.len);
+ 	strbuf_release(&line);
+ 	cat_blob_write(buf, size);
+ 	cat_blob_write("\n", 1);
+ 	free(buf);
+ }
+ 
+ static void parse_cat_blob(void)
+ {
+ 	const char *p;
+ 	struct object_entry *oe = oe;
+ 	unsigned char sha1[20];
+ 
+ 	/* cat-blob SP <object> LF */
+ 	p = command_buf.buf + strlen("cat-blob ");
+ 	if (*p == ':') {
+ 		char *x;
+ 		oe = find_mark(strtoumax(p + 1, &x, 10));
+ 		if (x == p + 1)
+ 			die("Invalid mark: %s", command_buf.buf);
+ 		if (!oe)
+ 			die("Unknown mark: %s", command_buf.buf);
+ 		if (*x)
+ 			die("Garbage after mark: %s", command_buf.buf);
+ 		hashcpy(sha1, oe->idx.sha1);
+ 	} else {
+ 		if (get_sha1_hex(p, sha1))
+ 			die("Invalid SHA1: %s", command_buf.buf);
+ 		if (p[40])
+ 			die("Garbage after SHA1: %s", command_buf.buf);
+ 		oe = find_object(sha1);
+ 	}
+ 
+ 	cat_blob(oe, sha1);
+ }
+ 
 -static void parse_checkpoint(void)
 +static void checkpoint(void)
  {
 +	checkpoint_requested = 0;
  	if (object_count) {
  		cycle_packfile();
  		dump_branches();
  		dump_tags();
  		dump_marks();
  	}
 +}
 +
 +static void parse_checkpoint(void)
 +{
 +	checkpoint_requested = 1;
  	skip_optional_lf();
  }
  
@@@ -2802,16 -2832,25 +2889,25 @@@ static void option_date_format(const ch
  		die("unknown --date-format argument %s", fmt);
  }
  
+ static unsigned long ulong_arg(const char *option, const char *arg)
+ {
+ 	char *endptr;
+ 	unsigned long rv = strtoul(arg, &endptr, 0);
+ 	if (strchr(arg, '-') || endptr == arg || *endptr)
+ 		die("%s: argument must be a non-negative integer", option);
+ 	return rv;
+ }
+ 
  static void option_depth(const char *depth)
  {
- 	max_depth = strtoul(depth, NULL, 0);
+ 	max_depth = ulong_arg("--depth", depth);
  	if (max_depth > MAX_DEPTH)
  		die("--depth cannot exceed %u", MAX_DEPTH);
  }
  
  static void option_active_branches(const char *branches)
  {
- 	max_active_branches = strtoul(branches, NULL, 0);
+ 	max_active_branches = ulong_arg("--active-branches", branches);
  }
  
  static void option_export_marks(const char *marks)
@@@ -2820,6 -2859,14 +2916,14 @@@
  	safe_create_leading_directories_const(export_marks_file);
  }
  
+ static void option_cat_blob_fd(const char *fd)
+ {
+ 	unsigned long n = ulong_arg("--cat-blob-fd", fd);
+ 	if (n > (unsigned long) INT_MAX)
+ 		die("--cat-blob-fd cannot exceed %d", INT_MAX);
+ 	cat_blob_fd = (int) n;
+ }
+ 
  static void option_export_pack_edges(const char *edges)
  {
  	if (pack_edges)
@@@ -2873,6 -2920,8 +2977,8 @@@ static int parse_one_feature(const cha
  		option_import_marks(feature + 13, from_stream);
  	} else if (!prefixcmp(feature, "export-marks=")) {
  		option_export_marks(feature + 13);
+ 	} else if (!strcmp(feature, "cat-blob")) {
+ 		; /* Don't die - this feature is supported */
  	} else if (!prefixcmp(feature, "relative-marks")) {
  		relative_marks_paths = 1;
  	} else if (!prefixcmp(feature, "no-relative-marks")) {
@@@ -2967,6 -3016,11 +3073,11 @@@ static void parse_argv(void
  		if (parse_one_feature(a + 2, 0))
  			continue;
  
+ 		if (!prefixcmp(a + 2, "cat-blob-fd=")) {
+ 			option_cat_blob_fd(a + 2 + strlen("cat-blob-fd="));
+ 			continue;
+ 		}
+ 
  		die("unknown option %s", a);
  	}
  	if (i != global_argc)
@@@ -3009,7 -3063,6 +3120,7 @@@ int main(int argc, const char **argv
  	prepare_packed_git();
  	start_packfile();
  	set_die_routine(die_nicely);
 +	set_checkpoint_signal();
  	while (read_next_command() != EOF) {
  		if (!strcmp("blob", command_buf.buf))
  			parse_new_blob();
@@@ -3031,9 -3084,6 +3142,9 @@@
  			/* ignore non-git options*/;
  		else
  			die("Unsupported command: %s", command_buf.buf);
 +
 +		if (checkpoint_requested)
 +			checkpoint();
  	}
  
  	/* argv hasn't been parsed yet, do so */
diff --combined t/t9300-fast-import.sh
index e8034d410f,ed28d3cc83..5a1925f690
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@@ -23,11 -23,18 +23,18 @@@ file5_data='an inline file
  file6_data='#!/bin/sh
  echo "$@"'
  
+ >empty
+ 
  ###
  ### series A
  ###
  
  test_tick
+ 
+ test_expect_success 'empty stream succeeds' '
+ 	git fast-import </dev/null
+ '
+ 
  cat >input <<INPUT_END
  blob
  mark :2
@@@ -928,114 -935,6 +935,114 @@@ test_expect_success 
  	 git diff-tree -C --find-copies-harder -r N5^^ N5 >actual &&
  	 compare_diff_raw expect actual'
  
 +test_expect_success \
 +	'N: reject foo/ syntax' \
 +	'subdir=$(git rev-parse refs/heads/branch^0:file2) &&
 +	 test_must_fail git fast-import <<-INPUT_END
 +	commit refs/heads/N5B
 +	committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
 +	data <<COMMIT
 +	copy with invalid syntax
 +	COMMIT
 +
 +	from refs/heads/branch^0
 +	M 040000 $subdir file3/
 +	INPUT_END'
 +
 +test_expect_success \
 +	'N: copy to root by id and modify' \
 +	'echo "hello, world" >expect.foo &&
 +	 echo hello >expect.bar &&
 +	 git fast-import <<-SETUP_END &&
 +	commit refs/heads/N7
 +	committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
 +	data <<COMMIT
 +	hello, tree
 +	COMMIT
 +
 +	deleteall
 +	M 644 inline foo/bar
 +	data <<EOF
 +	hello
 +	EOF
 +	SETUP_END
 +
 +	 tree=$(git rev-parse --verify N7:) &&
 +	 git fast-import <<-INPUT_END &&
 +	commit refs/heads/N8
 +	committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
 +	data <<COMMIT
 +	copy to root by id and modify
 +	COMMIT
 +
 +	M 040000 $tree ""
 +	M 644 inline foo/foo
 +	data <<EOF
 +	hello, world
 +	EOF
 +	INPUT_END
 +	 git show N8:foo/foo >actual.foo &&
 +	 git show N8:foo/bar >actual.bar &&
 +	 test_cmp expect.foo actual.foo &&
 +	 test_cmp expect.bar actual.bar'
 +
 +test_expect_success \
 +	'N: extract subtree' \
 +	'branch=$(git rev-parse --verify refs/heads/branch^{tree}) &&
 +	 cat >input <<-INPUT_END &&
 +	commit refs/heads/N9
 +	committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
 +	data <<COMMIT
 +	extract subtree branch:newdir
 +	COMMIT
 +
 +	M 040000 $branch ""
 +	C "newdir" ""
 +	INPUT_END
 +	 git fast-import <input &&
 +	 git diff --exit-code branch:newdir N9'
 +
 +test_expect_success \
 +	'N: modify subtree, extract it, and modify again' \
 +	'echo hello >expect.baz &&
 +	 echo hello, world >expect.qux &&
 +	 git fast-import <<-SETUP_END &&
 +	commit refs/heads/N10
 +	committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
 +	data <<COMMIT
 +	hello, tree
 +	COMMIT
 +
 +	deleteall
 +	M 644 inline foo/bar/baz
 +	data <<EOF
 +	hello
 +	EOF
 +	SETUP_END
 +
 +	 tree=$(git rev-parse --verify N10:) &&
 +	 git fast-import <<-INPUT_END &&
 +	commit refs/heads/N11
 +	committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
 +	data <<COMMIT
 +	copy to root by id and modify
 +	COMMIT
 +
 +	M 040000 $tree ""
 +	M 100644 inline foo/bar/qux
 +	data <<EOF
 +	hello, world
 +	EOF
 +	R "foo" ""
 +	C "bar/qux" "bar/quux"
 +	INPUT_END
 +	 git show N11:bar/baz >actual.baz &&
 +	 git show N11:bar/qux >actual.qux &&
 +	 git show N11:bar/quux >actual.quux &&
 +	 test_cmp expect.baz actual.baz &&
 +	 test_cmp expect.qux actual.qux &&
 +	 test_cmp expect.qux actual.quux'
 +
  ###
  ### series O
  ###
@@@ -1740,6 -1639,256 +1747,256 @@@ test_expect_success 'R: feature no-rela
      test_cmp marks.new non-relative.out
  '
  
+ test_expect_success 'R: feature cat-blob supported' '
+ 	echo "feature cat-blob" |
+ 	git fast-import
+ '
+ 
+ test_expect_success 'R: cat-blob-fd must be a nonnegative integer' '
+ 	test_must_fail git fast-import --cat-blob-fd=-1 </dev/null
+ '
+ 
+ test_expect_success 'R: print old blob' '
+ 	blob=$(echo "yes it can" | git hash-object -w --stdin) &&
+ 	cat >expect <<-EOF &&
+ 	${blob} blob 11
+ 	yes it can
+ 
+ 	EOF
+ 	echo "cat-blob $blob" |
+ 	git fast-import --cat-blob-fd=6 6>actual &&
+ 	test_cmp expect actual
+ '
+ 
+ test_expect_success 'R: in-stream cat-blob-fd not respected' '
+ 	echo hello >greeting &&
+ 	blob=$(git hash-object -w greeting) &&
+ 	cat >expect <<-EOF &&
+ 	${blob} blob 6
+ 	hello
+ 
+ 	EOF
+ 	git fast-import --cat-blob-fd=3 3>actual.3 >actual.1 <<-EOF &&
+ 	cat-blob $blob
+ 	EOF
+ 	test_cmp expect actual.3 &&
+ 	test_cmp empty actual.1 &&
+ 	git fast-import 3>actual.3 >actual.1 <<-EOF &&
+ 	option cat-blob-fd=3
+ 	cat-blob $blob
+ 	EOF
+ 	test_cmp empty actual.3 &&
+ 	test_cmp expect actual.1
+ '
+ 
+ test_expect_success 'R: print new blob' '
+ 	blob=$(echo "yep yep yep" | git hash-object --stdin) &&
+ 	cat >expect <<-EOF &&
+ 	${blob} blob 12
+ 	yep yep yep
+ 
+ 	EOF
+ 	git fast-import --cat-blob-fd=6 6>actual <<-\EOF &&
+ 	blob
+ 	mark :1
+ 	data <<BLOB_END
+ 	yep yep yep
+ 	BLOB_END
+ 	cat-blob :1
+ 	EOF
+ 	test_cmp expect actual
+ '
+ 
+ test_expect_success 'R: print new blob by sha1' '
+ 	blob=$(echo "a new blob named by sha1" | git hash-object --stdin) &&
+ 	cat >expect <<-EOF &&
+ 	${blob} blob 25
+ 	a new blob named by sha1
+ 
+ 	EOF
+ 	git fast-import --cat-blob-fd=6 6>actual <<-EOF &&
+ 	blob
+ 	data <<BLOB_END
+ 	a new blob named by sha1
+ 	BLOB_END
+ 	cat-blob $blob
+ 	EOF
+ 	test_cmp expect actual
+ '
+ 
+ test_expect_success 'setup: big file' '
+ 	(
+ 		echo "the quick brown fox jumps over the lazy dog" >big &&
+ 		for i in 1 2 3
+ 		do
+ 			cat big big big big >bigger &&
+ 			cat bigger bigger bigger bigger >big ||
+ 			exit
+ 		done
+ 	)
+ '
+ 
+ test_expect_success 'R: print two blobs to stdout' '
+ 	blob1=$(git hash-object big) &&
+ 	blob1_len=$(wc -c <big) &&
+ 	blob2=$(echo hello | git hash-object --stdin) &&
+ 	{
+ 		echo ${blob1} blob $blob1_len &&
+ 		cat big &&
+ 		cat <<-EOF
+ 
+ 		${blob2} blob 6
+ 		hello
+ 
+ 		EOF
+ 	} >expect &&
+ 	{
+ 		cat <<-\END_PART1 &&
+ 			blob
+ 			mark :1
+ 			data <<data_end
+ 		END_PART1
+ 		cat big &&
+ 		cat <<-\EOF
+ 			data_end
+ 			blob
+ 			mark :2
+ 			data <<data_end
+ 			hello
+ 			data_end
+ 			cat-blob :1
+ 			cat-blob :2
+ 		EOF
+ 	} |
+ 	git fast-import >actual &&
+ 	test_cmp expect actual
+ '
+ 
+ test_expect_success 'setup: have pipes?' '
+ 	rm -f frob &&
+ 	if mkfifo frob
+ 	then
+ 		test_set_prereq PIPE
+ 	fi
+ '
+ 
+ test_expect_success PIPE 'R: copy using cat-file' '
+ 	expect_id=$(git hash-object big) &&
+ 	expect_len=$(wc -c <big) &&
+ 	echo $expect_id blob $expect_len >expect.response &&
+ 
+ 	rm -f blobs &&
+ 	cat >frontend <<-\FRONTEND_END &&
+ 	#!/bin/sh
+ 	cat <<EOF &&
+ 	feature cat-blob
+ 	blob
+ 	mark :1
+ 	data <<BLOB
+ 	EOF
+ 	cat big
+ 	cat <<EOF
+ 	BLOB
+ 	cat-blob :1
+ 	EOF
+ 
+ 	read blob_id type size <&3 &&
+ 	echo "$blob_id $type $size" >response &&
+ 	dd of=blob bs=1 count=$size <&3 &&
+ 	read newline <&3 &&
+ 
+ 	cat <<EOF &&
+ 	commit refs/heads/copied
+ 	committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+ 	data <<COMMIT
+ 	copy big file as file3
+ 	COMMIT
+ 	M 644 inline file3
+ 	data <<BLOB
+ 	EOF
+ 	cat blob &&
+ 	cat <<EOF
+ 	BLOB
+ 	EOF
+ 	FRONTEND_END
+ 
+ 	mkfifo blobs &&
+ 	(
+ 		export GIT_COMMITTER_NAME GIT_COMMITTER_EMAIL GIT_COMMITTER_DATE &&
+ 		sh frontend 3<blobs |
+ 		git fast-import --cat-blob-fd=3 3>blobs
+ 	) &&
+ 	git show copied:file3 >actual &&
+ 	test_cmp expect.response response &&
+ 	test_cmp big actual
+ '
+ 
+ test_expect_success PIPE 'R: print blob mid-commit' '
+ 	rm -f blobs &&
+ 	echo "A blob from _before_ the commit." >expect &&
+ 	mkfifo blobs &&
+ 	(
+ 		exec 3<blobs &&
+ 		cat <<-EOF &&
+ 		feature cat-blob
+ 		blob
+ 		mark :1
+ 		data <<BLOB
+ 		A blob from _before_ the commit.
+ 		BLOB
+ 		commit refs/heads/temporary
+ 		committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+ 		data <<COMMIT
+ 		Empty commit
+ 		COMMIT
+ 		cat-blob :1
+ 		EOF
+ 
+ 		read blob_id type size <&3 &&
+ 		dd of=actual bs=1 count=$size <&3 &&
+ 		read newline <&3 &&
+ 
+ 		echo
+ 	) |
+ 	git fast-import --cat-blob-fd=3 3>blobs &&
+ 	test_cmp expect actual
+ '
+ 
+ test_expect_success PIPE 'R: print staged blob within commit' '
+ 	rm -f blobs &&
+ 	echo "A blob from _within_ the commit." >expect &&
+ 	mkfifo blobs &&
+ 	(
+ 		exec 3<blobs &&
+ 		cat <<-EOF &&
+ 		feature cat-blob
+ 		commit refs/heads/within
+ 		committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+ 		data <<COMMIT
+ 		Empty commit
+ 		COMMIT
+ 		M 644 inline within
+ 		data <<BLOB
+ 		A blob from _within_ the commit.
+ 		BLOB
+ 		EOF
+ 
+ 		to_get=$(
+ 			echo "A blob from _within_ the commit." |
+ 			git hash-object --stdin
+ 		) &&
+ 		echo "cat-blob $to_get" &&
+ 
+ 		read blob_id type size <&3 &&
+ 		dd of=actual bs=1 count=$size <&3 &&
+ 		read newline <&3 &&
+ 
+ 		echo deleteall
+ 	) |
+ 	git fast-import --cat-blob-fd=3 3>blobs &&
+ 	test_cmp expect actual
+ '
+ 
  cat >input << EOF
  option git quiet
  blob
@@@ -1748,8 -1897,6 +2005,6 @@@ h
  
  EOF
  
- touch empty
- 
  test_expect_success 'R: quiet option results in no stats being output' '
      cat input | git fast-import 2> output &&
      test_cmp empty output
@@@ -1767,6 -1914,14 +2022,14 @@@ test_expect_success 'R: unknown command
      test_must_fail git fast-import --non-existing-option < /dev/null
  '
  
+ test_expect_success 'R: die on invalid option argument' '
+ 	echo "option git active-branches=-5" |
+ 	test_must_fail git fast-import &&
+ 	echo "option git depth=" |
+ 	test_must_fail git fast-import &&
+ 	test_must_fail git fast-import --depth="5 elephants" </dev/null
+ '
+ 
  cat >input <<EOF
  option non-existing-vcs non-existing-option
  EOF