git-pickaxe: optimize by avoiding repeated read_sha1_file().
It turns out that pickaxe reads the same blob repeatedly while
blame can reuse the blob already read for the parent when
handling a child commit when it's parent's turn to pass its
blame to the grandparent. Have a cache in the origin structure
to keep the blob there, which will be garbage collected when the
origin loses the last reference to it.
When we introduced the cached origin per commit, we gave up proper
garbage collecting because it meant that commits hold onto their
cached copy. There is no need to do so.
git-pickaxe: re-scan the blob after making progress with -C
The reason to do this is the same as in the previous change for
line copy detection within the same file (-M).
Also this fixes -C and -C -C (aka find-copies-harder) logic; in
this application we are not interested in the similarity
matching diffcore-rename makes, because we are only interested
in scanning files that were modified, or in the case of -C -C,
scanning all files in the parent and we want to do that
ourselves.
git-pickaxe: re-scan the blob after making progress with -M
Otherwise we would miss copied lines that are contained in the
parts before or after the part that we find after splitting the
blame_entry (i.e. split[0] and split[2]).
This is a shorthand for "<rev> --not <rev>^@", i.e. "include
this commit but exclude any of its parents".
When a new file $F is introduced by revision $R, this notation
can be used to find a copy-and-paste from existing file in the
parents of that revision without annotating the ancestry of the
lines that were copied from:
git-pickaxe: cache one already found path per commit.
Depending on how bushy the commit DAG is, this saves calls to
the internal diff-tree for fork-point commits. For example,
annotating Makefile in the kernel repository saves about a third
of such diff-tree calls.
git-pickaxe: split find_origin() into find_rename() and find_origin().
When a merge adds a new file from the second parent, the
earlier code tried to find renames in the first parent before
noticing that the vertion from the second parent was added
without modification.
The origin structure is allocated for each commit and path while
the code traverse down it is copied into different blame entries.
To avoid leaks, try refcounting them.
This still seems to leak, which I haven't tracked down fully yet.
When assigning blames for code movements across file boundaries,
we used to iterate over blame entries (i.e. groups of lines to
be blamed) in the outer loop and compared each entry with paths
in the parent commit in an inner loop. This meant that we
opened the blob data from each path number of times.
Reorganize the loop so that we read the same path only once, and
compare it against all relevant blame entries.
This should perform better, but seems to give mixed results,
though.
After finding out which path in the parent to scan to pass
blames, using get_tree_entry() to extract the blob information
again was quite wasteful, since diff-tree already gave us that
information. Separate the function to create an origin out as
get_origin().
You'll never know what is more efficient unless you try and/or
think hard. I somehow thought that extracting one known path
out of commit's tree is cheaper than running a diff-tree for the
current path between the commit and its parent, but it is not
the case. In real, non-toy projects, most commits do not touch
the path you are interested in, and if the path is a few levels
away from the toplevel, whole-subdirectory comparison logic
diff-tree allows us to skip opening lower subdirectories.
This commit rewrites find_origin() function to use a single-path
diff-tree to see if the parent has the same blob as the current
suspect, which is cheaper than extracting the blob information
using get_tree_entry() and comparing it with what the current
suspect has. This shaves about 6% overhead when annotating
kernel/sched.c in the Linux kernel repository on my machine.
The saving rises to 25% for arch/i386/kernel/Makefile.
git-pickaxe: do not confuse two origins that are the same.
It used to be that we can compare the address of the origin
structure to determine if they are the same because they are
always registered with scoreboard. After introduction of the
loop to try finding the best split, that is not true anymore.
The current code has rather serious leaks with origin structure,
but more importantly it gets confused when two origins that
points at the same commit and same path.
We might eventually have to refcount and gc origin, but let's
fix the correctness issue first.
git-pickaxe: introduce heuristics to avoid "trivial" chunks
This adds scoring logic to blame_entry to prevent blames on very
trivial chunks (e.g. lots of empty lines, indent followed by a
closing brace) from being passed down to unrelated lines in the
parent.
The current heuristics are quite simple and may need to be
tweaked later, but we need to start somewhere.
Instead of comparing number of lines matched, look at the
matched characters and count alnums, so that we do not pass
blame on not-so-interesting lines, such as an empty line and
a line that is indentation followed by a closing brace.
Add an option --score-debug to show the score of each
blame_entry while we cook this further on the "next" branch.
This completes the initial round of git-pickaxe. In addition to
the detection of line movements we already have, this finds new
lines that were created by moving or cutting-and-pasting lines
from different files in the parent.
With this,
git pickaxe -f -n -C v1.4.0 -- revision.c
finds that a major part of that file actually came from
rev-list.c when Linus split the latter at commit ae563642 and
blames them to earlier commits that touch rev-list.c.
git-pickaxe -M: blame line movements within a file.
This makes pickaxe more intelligent than the classic blame.
A typical example is a change that moves one static C function
from lower part of the file to upper part of the same file,
because you added a new caller in the middle.
The versions in the parent and the child would look like this:
parent child
A static foo() {
B ...
C }
D A
E B
F C
G D
static foo() { ... call foo();
... E
} F
H G
H
With the classic blame algorithm, we can blame lines A B C D E F
G and H to the parent. The child is guilty of introducing the
line "... call foo();", and the blame is placed on the child.
However, the classic blame algorithm fails to notice that the
implementation of foo() at the top of the file is not new, and
moved from the lower part of the parent.
This commit introduces detection of such line movements, and
correctly blames the lines that were simply moved in the file to
the parent.
Currently it does what git-blame does, but only faster.
More importantly, its internal structure is designed to support
content movement (aka cut-and-paste) more easily by allowing
more than one paths to be taken from the same commit.
git-apply: prepare for upcoming GNU diff -u format change.
The latest GNU diff from CVS emits an empty line to express
an empty context line, instead of more traditional "single
white space followed by a newline". Do not get broken by it.
Signed-off-by: Linus Torvalds <torvalds@osdl.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Don't use $author_name undefined when $from contains no /\s</.
I noticed a case not handled in a recent patch.
Demonstrate it like this:
$ touch new-file
$ git-send-email --dry-run --from j --to k new-file 2>err
new-file
OK. Log says:
Date: Thu, 19 Oct 2006 10:26:24 +0200
Sendmail: /usr/sbin/sendmail
From: j
Subject:
Cc:
To: k
Result: OK
$ cat err
Use of uninitialized value in pattern match (m//) at /p/bin/git-send-email line 416.
Use of uninitialized value in concatenation (.) or string at /p/bin/git-send-email line 420.
Use of uninitialized value in concatenation (.) or string at /p/bin/git-send-email line 468.
There's a patch for the $author_name part below.
The example above shows that $subject may also be used uninitialized.
That should be easy to fix, too.
Signed-off-by: Jim Meyering <jim@meyering.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
* mw/pathinfo:
gitweb: Fix search form when PATH_INFO is enabled
gitweb: Document features better
gitweb: warn if feature cannot be overridden.
gitweb: start to generate PATH_INFO URLs.
* jc/send-email:
Make git-send-email detect mbox-style patches more readily
git-send-email: real name with period need to be dq-quoted on From: line
git-send-email: do not drop custom headers the user prepared
* rs/rebase:
git-rebase: Add a -v option to show a diffstat of the changes upstream at the start of a rebase.
git-rebase: Use --ignore-if-in-upstream option when executing git-format-patch.
Supposing that both the base and result sizes were both full size 64-bit
values, their encoding would occupy only 9.2 bytes each. Therefore
inflating 64 bytes is way overkill. Limit it to 20 bytes instead which
should be plenty enough for a couple years to come.
Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
v1.2.0 is Ok and I personally would say it is old enough, but
the improvement between version 2 and version 3 delta is not
bit enough to justify breaking older clients.
We should resurrect this later, but when we do so we shold
make it conditional.
clone: the given repository dir should be relative to $PWD
the repository argument for git-clone should be relative to $PWD
instead of the given target directory. The old behavior gave us
surprising success and you need a few minute to know why it worked.
GIT_DIR is already exported so no need to cd into $D. And this makes
$PWD for git-fetch-pack, which is the actual command to take the given
repository dir, the same as git-clone.
Signed-off-by: Yasushi SHOJI <yashi@atmark-techno.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
apply_textdelta and send_stream can use a separate pool from the
rest of the editor interface, so we'll use a separate SVN::Pool
for them and clear the pool after each file is sent to SVN.
This drastically reduces memory usage per-changeset committed,
and makes large commits (and initial imports) of several
thousand files possible.
Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
Most callers of write_sha1_file_prepare() are only interested in the
resulting hash but don't care about the returned file name or the header.
This patch adds a simple wrapper named hash_sha1_file() which does just
that, and converts potential callers.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <junkio@cox.net>
Once a get_commit_editor has been called from an SVN session, RA
layer operations are not allowed (well, unless you're using
file:// or http(s)://). So we'll pass an alternate SVN::Ra
object to our editor object for running 'check-path'.
This should fix commits over svnserve (svn:// without ssh, too).
Closes Debian bug #392702, thanks to Pierre Habouzit for
reporting the bug.
Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
When whitespace or whitespace change was ignored, the function
xdl_recmatch() returned memcmp() style differences, which is wrong,
since it should return 0 on non-match.
Also, there were three horrible off-by-one bugs, even leading to wrong
hashes in the whitespace special handling.
The issue was noticed by Ray Lehtiniemi.
For good measure, this commit adds a test.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <junkio@cox.net>
git-svn: add a message encouraging use of SVN::* libraries
I'm using svn 1.4.0-4 in Debian unstable and apparently there's
a regression on the SVN side that prevents a symlink from
becoming a regular file (which git supports, of course).
It's not a noticeable regression for most people, but this broke
the full-svn-tests target in t/Makefile for me.
The SVN::* Perl libraries seem to have matured and improved over
the past year, and git-svn has supported them for several months
now, so with that I encourage all users to start using the
SVN::* Perl libraries with git-svn.
Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
commit: fix a segfault when displaying a commit with unreachable parents
I was running git show on various commits found by fsck-objects
when I found this bug. Since find_unique_abbrev() cannot find
an abbreviation for an object not in the database, it will
return NULL, which is bad to run strlen() on. So instead, we'll
just display the unabbreviated sha1 that we referenced in the
commit.
I'm not sure that this is the best 'fix' for it because the
commit I was trying to show was broken, but I don't think a
program should segfault even if the user tries to do something
stupid.
Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
'graft-branches' is slightly longer than the rest of the
commands, so the text was squished together in the formatted
output. This patch just adds some more whitespace to make
the text look more pleasant.
Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
Documentation/git-svn: document some of the newer features
I've forgotten to document many of the features added along the
way in the manpages. This fills in some holes in the
documentation and adds updates some outdated information.
Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
Currently that was broken. Ideal fix would make the search form use
PATH_INFO too, but it's just one insignificant place so it's no big deal if
we don't for now... This at least makes it work.
Signed-off-by: Petr Baudis <pasky@suse.cz> Signed-off-by: Junio C Hamano <junkio@cox.net>
Change the --verbose flag to more closely match svn. I was
somehow under the impression that --summary included --raw diff
output, but I was wrong. We now pass -r --raw --name-status as
arguments if passed -v/--verbose.
-r (recursive) is passed by default, since users usually want
it, and accepting it causes difficulty with the -r<revision>
option used by svn users. A --non-recursive switch has been
added to disable this.
Of course, --summary, --raw, -p and any other git-log options
can still be passed directly (without --name-status).
Also, several warnings about referencing undefined variables
have been fixed.
Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
git-svn: multi-init saves and reuses --tags and --branches arguments
This should make it much easier to track newly added tags and
branches. Re-running multi-init without command-line arguments
should now detect new-tags and branches.
--trunk shouldn't change often, but running multi-init on it
is now idempotent.
Signed-off-by: Eric Wong <normalperson@yhbt.net> Signed-off-by: Junio C Hamano <junkio@cox.net>
The confusion can be avoided in most cases by writing the remote message
in one go to prevent interleacing with local messages. The buffer
declaration has been moved inside recv_sideband() to avoid extra string
copies.
Signed-off-by: Nicolas Pitre <nico@cam.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Add a --dry-run option to git-send-email due to having made too many
mistakes with it in the past week. I like having a safety catch on my
machine gun.
Signed-off-by: Matthew @ilcox <matthew@wil.cx> Signed-off-by: Junio C Hamano <junkio@cox.net>
Martin Waitz noticed that one of the case arms had an impossible
choice. It turns out that what it was checking was redundant and
the typo did not have any effect.
git-svnimport.perl: copying directory from original SVN place
When copying whole directory, if source directory is not in already
imported tree, try to get it from original SVN location. This happens
when source directory is not matched by provided 'trunk' and/or
'tags/branches' templates or when it is not part of specified SVN
sub-project.
Signed-off-by: Sasha Khapyorsky <sashak@voltaire.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
> On Sat, 07 Oct 2006 21:52:02 -0700
> Junio C Hamano <junkio@cox.net> wrote:
>
>> Using DAV, if it works with the server, has the advantage of not
>> having to keep objects/info/packs up-to-date from repository
>> owner's point of view. But the repository owner ends up keeping
>> up-to-date as a side effect of keeping info/refs up-to-date
>> anyway (as I do not see a code to read that information over
>> DAV), so there is no point doing this over DAV in practice.
>>
>> Perhaps we should remove call to remote_ls() from
>> fetch_indices() unconditionally, not just protected with
>> NO_EXPAT and be done with it?
>
> That makes a lot of sense. A server really has to always provide
> a objects/info/packs anyway, just to be fetchable today by clients
> that are compiled with NO_EXPAT.
And even for an isolated group where everybody knows that
everybody else runs DAV-enabled clients, they need info/refs
prepared for ls-remote and git-fetch script, which means you
will run update-server-info to keep objects/info/packs up to
date.
Nick, do you see holes in my logic?
-- >8 --
http-fetch.c: drop remote_ls()
While doing remote_ls() over DAV potentially allows the server
side not to keep objects/info/pack up-to-date, misconfigured or
buggy servers can silently ignore or not to respond to DAV
requests and makes the client hang.
The server side (unfortunately) needs to run git-update-server-info
even if remote_ls() removes the need to keep objects/info/pack file
up-to-date, because the caller of git-http-fetch (git-fetch) and other
clients that interact with the repository (e.g. git-ls-remote) need to
read from info/refs file (there is no code to make that unnecessary by
using DAV yet).
Perhaps the right solution in the longer-term is to make info/refs
also unnecessary by using DAV, and we would want to resurrect the
code this patch removes when we do so, but let's drop remote_ls()
implementation for now. It is causing problems without really
helping anything yet.
git will keep it for us until we need it next time.
gitweb: Cleanup Git logo and Git logo target generation
Rename $githelp_url and $githelp_label to $logo_url and $logo_label to
be more obvious what they refer to; while at it add commented out
previous contents (git documentation at kernel.org). Add comment about
logo size.
Use $cgi->a(...) to generate Git logo link; it automatically escapes
attribute values when it is needed. Escape href attribute using
esc_url instead of (incorrect!) esc_html.
Move styling of git logo <img> element from "style" attribute to CSS
via setting class to "logo". Perhaps we should set it by id rather
than by class.
Signed-off-by: Jakub Narebski <jnareb@gmail.com> Signed-off-by: Junio C Hamano <junkio@cox.net>
Noted by Jiri Slaby, git-tar-tree --remote doesn't need to be run
from inside of a git archive. Since git-tar-tree is now only a
wrapper for git-archive, which calls setup_git_directory() as
needed, we should drop the flag RUN_SETUP.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <junkio@cox.net>
Make git-send-email detect mbox-style patches more readily
Earlier we insisted that mbox file to begin with "From ". That
is fine as long as you feed format-patch output, but if you
handcraft the input file, this is unnecessary burden. We should
detect lines that look like e-mail headers and say that is also
a mbox file.
The other input file format is traditional "send lots of email",
whose first line would never look like e-mail headers, so this
is a safe change.
The original patch was done by Matthew Wilcox, which checked
explicitly for headers the script pays attention to.
Add symlink support to ZIP file creation, and a few tests.
This implementation sets the "version made by" field
(creator_version) to Unix for symlinks, only; regular files and
directories are still marked as originating from FAT/VFAT/NTFS.
Also set "external file attributes" (attr2) to 0 for regular
files and 16 for directories (FAT attribute), and to the file
mode for symlinks.
We could always set the creator_version to Unix and include the
mode, but then Info-ZIP unzip would set the mode of the extracted
files to *exactly* the value stored in attr2. The FAT trick
makes it apply the umask instead. Note: FAT has no executable
bit, so this information is not stored in the ZIP file.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <junkio@cox.net>
Use 10 for the "version needed to extract" field. This is the
default value, and we want to use it because we don't do anything
special. Info-ZIP's zip uses it, too.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx> Signed-off-by: Junio C Hamano <junkio@cox.net>
This expands gitweb/README to talk some more about GITWEB_CONFIG, moves
feature-specific documentation in gitweb.cgi to the inside of the %features
array, and adds some short description of all the features.
Signed-off-by: Petr Baudis <pasky@suse.cz> Signed-off-by: Junio C Hamano <junkio@cox.net>
test-lib: separate individual test better in verbose mode.
When running tests with --verbose it is difficult to see where
one test starts and where it ends because everything is printed
in one big lump.
Fix that by printing one single newline between each test.
Signed-off-by: Martin Waitz <tali@admingilde.org> Signed-off-by: Junio C Hamano <junkio@cox.net>
Add default values for --window and --depth to the docs
Currently, you actually have to read the source to find out the
default values. While at it, fix two typos and suggest that these
options actually take a parameter in git-pack-objects.txt.
Signed-off-by: Dennis Stosberg <dennis@stosberg.net> Signed-off-by: Junio C Hamano <junkio@cox.net>