This provides a linear decrement on the penalty related to delta depth
instead of being an 1/x function. With this another 5% reduction is
observed on packs for both the GIT repo and the Linux kernel repo, as
well as fixing a pack size regression in another sample repo I have.
Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
* jc/grep: (22 commits)
Fix silly typo in new builtin grep
builtin-grep: unparse more command line options.
builtin-grep: use external grep when we can take advantage of it
builtin-grep: -F (--fixed-strings)
builtin-grep: -w fix
builtin-grep: typofix
builtin-grep: tighten argument parsing.
builtin-grep: documentation
Teach -f <file> option to builtin-grep.
builtin-grep: -L (--files-without-match).
builtin-grep: binary files -a and -I
builtin-grep: terminate correctly at EOF
builtin-grep: tighten path wildcard vs tree traversal.
builtin-grep: support -w (--word-regexp).
builtin-grep: support -c (--count).
builtin-grep: allow more than one patterns.
builtin-grep: allow -<n> and -[ABC]<n> notation for context lines.
builtin-grep: printf %.*s length is int, not ptrdiff_t.
builtin-grep: do not use setup_revisions()
builtin-grep: support '-l' option.
...
* fix:
Fix pack-index issue on 64-bit platforms a bit more portably.
Install git-send-email by default
Fix compilation on newer NetBSD systems
git config syntax updates
Another config file parsing fix.
checkout: use --aggressive when running a 3-way merge (-m).
Fix pack-index issue on 64-bit platforms a bit more portably.
Apparently <stdint.h> is not enough for uint32_t on OpenBSD; use
"unsigned int" -- hopefully that would stay 32-bit on every
platform we care about, at least until we update the pack-index
file format.
Our sha1 routines optimized for architectures use uint32_t and
expects '#include <stdint.h>' to be enough, so OpenBSD on arm or
ppc might have similar issues down the road, I dunno.
Avoid creating a delta index for objects with maximum depth since they
are not going to be used as delta base anyway. This also reduce peak
memory usage slightly as the current object's delta index is not useful
until the next object in the loop is considered for deltification. This
saves a bit more than 1% on CPU usage.
Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
simple euristic for further free packing improvements
Given that the early eviction of objects with maximum delta depth
may exhibit bad packing on its own, why not considering a bias against
deep base objects in try_delta() to mitigate that bad behavior.
This patch adjust the MAX_size allowed for a delta based on the depth of
the base object as well as enabling the early eviction of max depth
objects from the object window. When used separately, those two things
produce slightly better and much worse results respectively. But their
combined effect is a surprising significant packing improvement.
With this really simple patch the GIT repo gets nearly 15% smaller, and
the Linux kernel repo about 5% smaller, with no significantly measurable
CPU usage difference.
Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
The previous commit makes -u to mean "I do want to remove the
local changes, just update it from the read tree" only for
one-way merge. It makes sense to have it depend on the
"--reset" flag instead.
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
send-email: quiet some warnings, reject invalid addresses
I'm not sure why we never actually rejected invalid addresses in
the first place. We just seemed to be using our email validity
checkers to kill duplicates.
Now we just drop invalid email addresses completely and warn
the user about it.
Since we support local sendmail, we'll also accept username-only
addresses.
Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
send-email: allow sendmail binary to be used instead of SMTP
This should make local mailing possible for machines without
a connection to an SMTP server.
It'll default to using /usr/sbin/sendmail or /usr/lib/sendmail
if no SMTP server is specified (the default). If it can't find
either of those paths, it'll fall back to connecting to an SMTP
server on localhost.
Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
read-tree -u one-way merge fix to check out locally modified paths.
The "-u" flag means "update the working tree files", but to
other types of merges, it also implies "I want to keep my local
changes" -- because they prevent local changes from getting lost
by using verify_uptodate. The one-way merge is different from
other merges in that its purpose is opposite of doing something
else while keeping unrelated local changes. The point of
one-way merge is to nuke local changes. So while it feels
somewhat wrong that this actively loses local changes, it is the
right thing to do.
builtin-grep: use external grep when we can take advantage of it
It's not perfect, but it gets the "git grep some-random-string" down to
the good old half-a-second range for the kernel.
It should convert more of the argument flags for "grep", that should be
trivial to expand (I did a few just as an example). It should also bother
to try to return the right "hit" value (which it doesn't, right now - the
code is kind of there, but I didn't actually bother to do it _right_).
Also, right now it _just_ limits by number of arguments, but it should
also strictly speaking limit by total argument size (ie add up the length
of the filenames, and do the "exec_grep()" flush call if it's bigger than
some random value like 32kB).
But I think that it's _conceptually_ doing all the right things, and it
seems to work. So maybe somebody else can do some of the final polish.
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
When renaming leading/a/filename to leading/b/filename (and
"filename" is sufficiently long), we tried to squash the rename
to "leading/{a => b}/filename". However, when "/a" or "/b" part
is empty, we underflowed and tried to print a substring of
length -1.
Now that the one-way merge strategy does the right thing wrt files that do
not exist in the result, just remove all the random crud we did in "git
reset" to do this all by hand.
Instead, just pass in "-u" to git-read-tree when we do a hard reset, and
depend on git-read-tree to update the working tree appropriately.
This basically means that git reset turns into
# Always update the HEAD ref
git update-ref HEAD "$rev"
case "--soft"
# do nothing to index/working tree
case "--hard"
# read index _and_ update working tree
git-read-tree --reset -u "$rev"
case "--mixed"
# update just index, report on working tree differences
git-read-tree --reset "$rev"
git-update-index --refresh
which is what it was always semantically doing, it just did it in a
rather strange way because it was written to not expect git-read-tree to
do anything to the working tree.
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
git diff: support "-U" and "--unified" options properly
We used to parse "-U" and "--unified" as part of the GIT_DIFF_OPTS
environment variable, but strangely enough we would _not_ parse them as
part of the normal diff command line (where we only accepted "-u").
This adds parsing of -U and --unified, both with an optional numeric
argument. So now you can just say
git diff --unified=5
to get a unified diff with a five-line context, instead of having to do
something silly like
GIT_DIFF_OPTS="--unified=5" git diff -u
(that silly format does continue to still work, of course).
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Add "--branches", "--tags" and "--remotes" options to git-rev-parse.
"git branch" uses "rev-parse --all" and becomes much too slow when
there are many tags (it scans all refs). Use the new "--branches"
option of rev-parse to speed things up.
Signed-off-by: Sean Estabrooks <seanlkml@sympatico.ca> Signed-off-by: Junio C Hamano <junkio@cox.net>
* ml/cvs:
Change to allow subdir updates from Eclipse
Many fixes for most operations in Eclipse.
Added logged warnings for CVS error returns
cvsserver: use git-rev-list instead of git-log
git-cvsexportcommit: Add -f(orce) and -m(essage prefix) flags, small cleanups.
* lt/fix-config:
git config syntax updates
Another config file parsing fix.
checkout: use --aggressive when running a 3-way merge (-m).
Fix git-pack-objects for 64-bit platforms
with manual adjustment of t/t1300 for "git repo-config --list" option.
This updates the hierarchical section name syntax to
[section<space>+"<randomstring>"]
where the only rule for "randomstring" is that it can't contain a newline,
and if you really want to insert a double-quote, you do it with \".
It turns that into the section name "secion.randomstring". The
"section" part is still case insensitive, but the "randomstring"
part is case sensitive.
So you could use this for things like
[email "torvalds@osdl.org"]
name = Linus Torvalds
if you wanted to do the "email->name" conversion as part of the config
file format (I'm not claiming that is sensible, I'm just giving it as an
insane example). That would show up as the association
email.torvalds@osdl.org.name -> Linus Torvalds
which is easy to parse (the "." in the email _looks_ ambiguous, but it
isn't: you know that there will always be a single key-name, so you find
the key name with "strrchr(name, '.')" and things are entirely
unambiguous).
Repo-config is updated to be able to parse the new format, and also
write things out in the new format.
[jc: rolled two patches from Linus and one fix-up from Sean into one,
with additional adjustments for t/t1300 test to check the case
insensitiveness of section base and variable and case sensitiveness
of the extended section part. Then stripped some part off to make
the result applicable to the stale 1.3.X series that does not have
recent enhancements. ]
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Sean Estabrooks <seanlkml@sympatico.ca> Signed-off-by: Junio C Hamano <junkio@cox.net>
If the variable we need to store should go into a section
that currently only has a single variable (not matching
the one we're trying to insert), we will already be into
the next section before we notice we've bypassed the correct
location to insert the variable.
To handle this case we store the current location as soon
as we find a variable matching the section of our new
variable.
This breakage was brought up by Linus.
Signed-off-by: Sean Estabrooks <seanlkml@sympatico.ca> Signed-off-by: Junio C Hamano <junkio@cox.net>
checkout: use --aggressive when running a 3-way merge (-m).
After doing an in-index 3-way merge, we always do the stock
"merge-index merge-one-file" without doing anything fancy;
use of --aggressive helps performance quite a bit.
The offset of an object in the pack is recorded as a 4-byte integer
in the index file. When reading the offset from the mmap'ed index
in prepare_pack_revindex(), the address is dereferenced as a long*.
This works fine as long as the long type is four bytes wide. On
NetBSD/sparc64, however, a long is 8 bytes wide and so dereferencing
the offset produces garbage.
[jc: taking suggestion by Linus to use uint32_t]
Signed-off-by: Dennis Stosberg <dennis@stosberg.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
It cannot be assumed that the given buffer will never be moved when
shrinking the allocated memory size with realloc(). So let's ignore
that optimization for now.
This patch makes Electric Fence happy on Linux.
Signed-off-by: Nicolas Pitre <nico@cam.org> Acked-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
checkout: use --aggressive when running a 3-way merge (-m).
After doing an in-index 3-way merge, we always do the stock
"merge-index merge-one-file" without doing anything fancy;
use of --aggressive helps performance quite a bit.
After doing an in-index 3-way merge, we always do the stock
"merge-index merge-one-file" without doing anything fancy;
use of --aggressive helps performance quite a bit.
* np/delta:
improve diff-delta with sparse and/or repetitive data
tiny optimization to diff-delta
replace adler32 with Rabin's polynomial in diff-delta
use delta index data when finding best delta matches
split the diff-delta interface
Merge branch 'tojunio' of locke.catalyst.net.nz/git/git-martinlanghoff into ml/cvs
* 'tojunio' of http://locke.catalyst.net.nz/git/git-martinlanghoff:
Change to allow subdir updates from Eclipse
Many fixes for most operations in Eclipse.
Added logged warnings for CVS error returns
cvsserver: use git-rev-list instead of git-log
git-cvsexportcommit: Add -f(orce) and -m(essage prefix) flags, small cleanups.
apply: fix infinite loop with multiple patches with --index
When multiple patches are passed to git-apply, it will attempt
to open multiple file descriptors to an index, which means
multiple entries will be in the circular cache_file_list.
This change makes git-apply only open the index once and
write the index at exit.
Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
This code is arguably pretty hot, if you use binary patches of course.
This patch helps gcc generate both smaller and faster code especially in
the error free path.
Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
When we cut off the front of a filename to make it fit on the line, we add
a "..." in front. However, the way the "git diff" code was written, we
will never reset the prefix back to the empty string, so every single
filename afterwards will have the "..." prefix, whether appropriate or
not.
You can see this with "git diff v2.6.16.." on the current kernel tree,
since there are filenames with long names that changed there:
notice how the two Documentation/firmware** filenames caused the "..." to
be added, but then the later filenames don't want it, and it also screws
up the alignment of the line numbering afterwards.
Trivially fixed by moving the declaration (and initial setting) of the
"prefix" variable into the for-loop where it is used.
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
When adding an alternate object store then add entries from its
info/alternates files, too.
Relative entries are only allowed in the current repository.
Loops and duplicate alternates through multiple repositories are ignored.
Just to be sure that nothing breaks it is not allow to build deep
nesting levels using info/alternates.
Signed-off-by: Martin Waitz <tali@admingilde.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
* fix:
repack: honor -d even when no new pack was created
clone: keep --reference even with -l -s
repo-config: document what value_regexp does a bit more clearly.
Release config lock if the regex is invalid
core-tutorial.txt: escape asterisk
* Implemented global -n option
* Implemented "Questionable"
* Fixed Directory method, I _believe_ it's now correct in both cmdline and Eclipse.
* Directory method Now looks for localdir of "." and compares the repo dir, uses THIS as a basis for all directory level calculations.
* Added extra parameter to filenamesplit() to force stripping of "prepended" directory name. This ensures commits/updates etc work from any directory in the source tree.
* Modified argsfromdir() so it is "always" called. This means that when the client specifies a directory, the method can detect this and behave accordingly (this is currently only implemented for the '.' directory)
* Fixed "commit" method to correctly work from in a subdir
Fix users of prefix_path() to free() only when necessary
Unfortunately, prefix_path() sometimes returns a newly xmalloc()ed buffer,
and in other cases it returns a substring!
For example, when calling
git update-index ./hello.txt
prefix_path() returns "hello.txt", but does not allocate a new buffer. The
original code only checked if the result of prefix_path() was different from
what was passed in, and thusly trigger a segmentation fault.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <junkio@cox.net>
Fix users of prefix_path() to free() only when necessary
Unfortunately, prefix_path() sometimes returns a newly xmalloc()ed buffer,
and in other cases it returns a substring!
For example, when calling
git update-index ./hello.txt
prefix_path() returns "hello.txt", but does not allocate a new buffer. The
original code only checked if the result of prefix_path() was different from
what was passed in, and thusly trigger a segmentation fault.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <junkio@cox.net>
If the variable we need to store should go into a section
that currently only has a single variable (not matching
the one we're trying to insert), we will already be into
the next section before we notice we've bypassed the correct
location to insert the variable.
To handle this case we store the current location as soon
as we find a variable matching the section of our new
variable.
This breakage was brought up by Linus.
Signed-off-by: Sean Estabrooks <seanlkml@sympatico.ca> Signed-off-by: Junio C Hamano <junkio@cox.net>