Add a svnrdump-simulator replaying a dump file for testing
To ease testing without depending on a reachable svn server, this
compact python script mimics parts of svnrdumps behaviour. It
requires the remote url to start with sim://.
Start and end revisions are evaluated. If the requested revision
doesn't exist, as it is the case with incremental imports, if no new
commit was added, it returns 1 (like svnrdump).
To allow using the same dump file for simulating multiple incremental
imports, the highest revision can be limited by setting the environment
variable SVNRMAX to that value. This simulates the situation where
higher revs don't exist yet.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Search for a note attached to the ref to update and read it's
'Revision-number:'-line. Start import from the next svn revision.
If there is no next revision in the svn repo, svnrdump terminates with
a message on stderr an non-zero return value. This looks a little
weird, but there is no other way to know whether there is a new
revision in the svn repo.
On the start of an incremental import, the parent of the first commit
in the fast-import stream is set to the branch name to update. All
following commits specify their parent by a mark number. Previous mark
files are currently not reused.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
remote-svn: Activate import/export-marks for fast-import
Enable import and export of a marks file by sending the appropriate
feature commands to fast-import before sending data.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Create a note for every imported commit containing svn metadata
To provide metadata from svn dumps for further processing, e.g.
branch detection, attach a note to each imported commit that stores
additional information. The notes are currently hard-coded in
refs/notes/svn/revs. Currently the following lines from the svn dump
are directly accumulated in the note. This can be refined as needed.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fast_export lacked a method to writes notes to fast-import stream.
Add two new functions fast_export_note which is similar to
fast_export_modify. And also add fast_export_buf_to_data to be able to
write inline blobs that don't come from a line_buffer or from delta
application.
Signed-off-by: Dmitry Ivankov <divanorama@gmail.com> Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Allow reading svn dumps from files via file:// urls
For testing as well as for importing large, already available dumps,
it's useful to bypass svnrdump and replay the svndump from a file
directly.
Add support for file:// urls in the remote url, e.g.
svn::file:///path/to/dump
When the remote helper finds an url starting with file:// it tries to
open that file instead of invoking svnrdump.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
remote-svn, vcs-svn: Enable fetching to private refs
The reference to update by the fast-import stream is hard-coded. When
fetching from a remote the remote-helper shall update refs in a
private namespace, i.e. a private subdir of refs/. This namespace is
defined by the 'refspec' capability, that the remote-helper advertises
as a reply to the 'capabilities' command.
Extend svndump and fast-export to allow passing the target ref.
Update svn-fe to be compatible.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
When debug==1, start fast-import with "--stats" instead of "--quiet"
fast-import prints statistics that could be interesting to the
developer of remote helpers.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add documentation for the 'bidi-import' capability of remote-helpers
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Connect fast-import to the remote-helper via pipe, adding 'bidi-import' capability
The fast-import commands 'cat-blob' and 'ls' can be used by
remote-helpers to retrieve information about blobs and trees that
already exist in fast-import's memory. This requires a channel from
fast-import to the remote-helper.
remote-helpers that use these features shall advertise the new
'bidi-import' capability to signal that they require the communication
channel. When forking fast-import in transport-helper.c connect it to
a dup of the remote-helper's stdin-pipe. The additional file
descriptor is passed to fast-import via its command line
(--cat-blob-fd). It follows that git and fast-import are connected to
the remote-helpers's stdin.
Because git can send multiple commands to the remote-helper on it's
stdin, it is required that helpers that advertise 'bidi-import' buffer
all input commands until the batch of 'import' commands is ended by a
newline before sending data to fast-import. This is to prevent mixing
commands and fast-import responses on the helper's stdin.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add argv_array_detach and argv_array_free_detached
Allow detaching of ownership of the argv_array's contents and add a
function to free those detached argv_arrays later.
This makes it possible to use argv_array efficiently with the exiting
struct child_process which only contains a member char **argv.
Add to documentation.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add svndump_init_fd to allow reading dumps from arbitrary FDs
The existing function only allows reading from a filename or from
stdin. Allow passing of a FD and an additional FD for the back report
pipe. This allows us to retrieve the name of the pipe in the caller.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The link-rule is a copy of the standard git$X rule but adds VCSSVN_LIB.
Add executable to .gitignore.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Enable basic fetching from subversion repositories. When processing
remote URLs starting with testsvn::, git invokes this remote-helper.
It starts svnrdump to extract revisions from the subversion repository
in the 'dump file format', and converts them to a git-fast-import stream
using the functions of vcs-svn/.
Imported refs are created in a private namespace at
refs/svn/<remote-name>/master. The revision history is imported
linearly (no branch detection) and completely, i.e. from revision 0 to
HEAD.
The 'bidi-import' capability is used. The remote-helper expects data
from fast-import on its stdin. It buffers a batch of 'import' command
lines in a string_list before starting to process them.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Acked-by: David Michael Barr <b@rr-dav.id.au> Signed-off-by: Junio C Hamano <gitster@pobox.com>
git svn: reset invalidates the memoized mergeinfo caches
Since v1.7.0-rc2~11 (git-svn: persistent memoization, 2010-01-30),
git-svn has maintained some private per-repository caches in
.git/svn/.caches to avoid refetching and recalculating some
mergeinfo-related information with every 'git svn fetch'.
This memoization can cause problems, e.g consider the following case:
SVN repo:
... - a - b - c - m <- trunk
\ /
d - e <- branch1
The Git import of the above repo is at commit 'a' and doesn't know about
the branch1. In case of an 'git svn rebase', only the trunk of the
SVN repo is imported. During the creation of the git commit 'm', git svn
uses the svn:mergeinfo property and tries to find the corresponding git
commit 'e' to create 'm' with 'c' and 'e' as parents. But git svn rebase
only imports the current branch so commit 'e' is not imported.
Therefore git svn fails to create commit 'm' as a merge commit, because one
of its parents is not known to git. The imported history looks like this:
... - a - b - c - m <- trunk
A later 'git svn fetch' to import all branches can't rewrite the commit 'm'
to add 'e' as a parent and to make it a real git merge commit, because it
was already imported.
That's why the imported history misses the merge and looks like this:
... - a - b - c - m <- trunk
\
d - e <- branch1
Right now the only known workaround for importing 'm' as a merge is to
force reimporting 'm' again from SVN, e.g. via
Sadly, this is where the behavior has regressed: git svn reset doesn't
invalidate the old mergeinfo cache, which is no longer valid for the
reimport, which leads to 'm' beeing imprted with only 'c' as parent.
As solution to this problem, this commit invalidates the mergeinfo cache
to force correct recalculation of the parents.
During development of this patch, several ways for invalidating the cache
where considered. One of them is to use Memoize::flush_cache, which will
call the CLEAR method on the underlying Memoize persistency implementation.
Sadly, neither Memoize::Storable nor the newer Memoize::YAML module
introduced in 68f532f4ba888 could optionally be used implement the
CLEAR method, so this is not an option.
Reseting the internal hash used to store the memoized values has the same
problem, because it calls the non-existing CLEAR method of the
underlying persistency layer, too.
Considering this and taking into account the different implementations
of the memoization modules, where Memoize::Storable is not in our control,
implementing the missing CLEAR method is not an option, at least not if
Memoize::Storable is still used.
Therefore the easiest solution to clear the cache is to delete the files
on disk in 'git svn reset'. Normally, deleting the files behind the back
of the memoization module would be problematic, because the in-memory
representation would still exist and contain wrong data. Fortunately, the
memoization is active in memory only for a small portion of the code.
Invalidating the cache by deleting the files on disk if it isn't active
should be safe.
Signed-off-by: Peter Baumann <waste.manager@gmx.de> Signed-off-by: Steven Walter <stevenrwalter@gmail.com> Signed-off-by: Eric Wong <normalperson@yhbt.net>
git svn: handle errors and concurrent commits in dcommit
dcommit didn't handle errors returned by SVN and coped very
poorly with concurrent commits that appear in SVN repository
while dcommit was running. In both cases it left git repository
in inconsistent state: index (which was reset with `git reset
--mixed' after a successful commit to SVN) no longer matched the
checkouted tree, when the following commit failed or needed to be
rebased. See http://bugs.debian.org/676904 for examples.
This patch fixes the issues by:
- introducing error handler for dcommit. The handler will try
to rebase or reset working tree before returning error to the
end user. dcommit_rebase function was extracted out of cmd_dcommit
to ensure consistency between cmd_dcommit and the error handler.
- calling `git reset --mixed' only once after all patches are
successfully committed to SVN. This ensures index is not touched
for most of the time of dcommit run.
* 'master' of git://github.com/git-l10n/git-po:
l10n: Update Swedish translation (1168t0f0u)
l10n: de.po: translate 77 new messages
l10n: vi.po: update one message
l10n: zh_CN.po: update one translation
l10n: Update one message in git.pot
Documentation: list git-credential in plumbing commands
Commit e30b2feb1b (Jun 24 2012, add 'git credential' plumbing command)
forgot to add git-credential to command-list.txt, hence the command was
not appearing in the documentation, making it hard for users to discover
it.
While we're there, capitalize the description line for git-crendential
for consistency with other commands.
Signed-off-by: Matthieu Moy <Matthieu.Moy@imag.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Translate 77 new messages came from git.pot update
in 3b6137f (l10n: Update git.pot (76 new, 4 removed
messages)) and bb2ba06 (l10n: Update one message in
git.pot).
Signed-off-by: Ralf Thielow <ralf.thielow@gmail.com> Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Now that git_terminal_prompt can cleanly interact with /dev/tty on
Solaris, enable HAVE_DEV_TTY so that this code path is used for
credential reading instead of relying on the crippled getpass().
Signed-off-by: Ben Walton <bwalton@artsci.utoronto.ca> Signed-off-by: Junio C Hamano <gitster@pobox.com>
terminal: seek when switching between reading and writing
When a stdio stream is opened in update mode (e.g., "w+"),
the C standard forbids switching between reading or writing
without an intervening positioning function. Many
implementations are lenient about this, but Solaris libc
will flush the recently-read contents to the output buffer.
In this instance, that meant writing the non-echoed password
that the user just typed to the terminal.
Fix it by inserting a no-op fseek between the read and
write.
The opposite direction (writing followed by reading) is also
disallowed, but our intervening fflush is an acceptable
positioning function for that alternative.
Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jn/block-sha1:
Makefile: BLK_SHA1 does not require fast htonl() and unaligned loads
block-sha1: put expanded macro parameters in parentheses
block-sha1: avoid pointer conversion that violates alignment constraints
* lm/git-blame-el:
git-blame.el: Do not use bare 0 to mean (point-min)
git-blame.el: Use with-current-buffer where appropriate
git-blame.el: Do not use goto-line in lisp code
merge-recursive: separate message for common ancestors
The function "merge_recursive" prints the count of common ancestors
as "found %u common ancestor(s):". We should use a singular and a
plural form of this message to help translators.
Signed-off-by: Ralf Thielow <ralf.thielow@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add i18n support for scripted Porcelains, and mark strings in
merge(-recursive), am, and rebase for i18n.
* jx/i18n-1.7.11:
i18n: merge-recursive: mark strings for translation
Remove dead code which contains bad gettext block
i18n: am: mark more strings for translation
rebase: remove obsolete and unused LONG_USAGE which breaks xgettext
i18n: Rewrite gettext messages start with dash
i18n: rebase: mark messages for translation
i18n: New keywords for xgettext extraction from sh
Merge branch 'jl/maint-1.7.10-recurse-submodules-with-symlink' into maint
When "git submodule add" clones a submodule repository, it can get
confused where to store the resulting submodule repository in the
superproject's .git/ directory when there is a symbolic link in the
path to the current directory.
* jl/maint-1.7.10-recurse-submodules-with-symlink:
submodules: don't stumble over symbolic links when cloning recursively
Merge branch 'jc/maint-filter-branch-epoch-date' into maint
In 1.7.9 era, we taught "git rebase" about the raw timestamp format
but we did not teach the same trick to "filter-branch", which rolled
a similar logic on its own.
* jc/maint-filter-branch-epoch-date:
t7003: add test to filter a branch with a commit at epoch
date.c: Fix off by one error in object-header date parsing
filter-branch: do not forget the '@' prefix to force git-timestamp
Add explanatory comment for transport-helpers refs mapping.
The patch below adds a comment to fetch_with_import() explaining the
loop that saves the fetched commit names after 'git fast-import' has
done its work. It avoids some confusion about which refs the
fast-import stream is supposed to use to write its result.
Signed-off-by: Florian Achleitner <florian.achleitner.2.6.31@gmail.com> Helped-by: Jonathan Nieder <jrnieder@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since commit bbc09c22 ("grep: rip out support for external grep",
12-01-2010), test number 60 ("grep -C1 hunk mark between files") is
essentially the same as test number 59.
Test 59 was intended to verify the behaviour of git-grep resulting
from multiple invocations of an external grep. As part of the test,
it creates and adds 1024 files to the index, which is now wasted
effort.
Remove test 59, since it is now redundant.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
In particular, the final test ('flags and then non flags') fails
intermittently, depending on how much time elapsed between the
invocations of "git commit-tree" when creating the commits which
later have their commit id's compared. For example, if the commits
for childid-3 and childid-4 are created 1 or more seconds apart,
then the commits, which would otherwise be identical, will have
different commit id's.
In order to make the test reproducible, we remove the variability
by setting the author and committer times to a well defined state.
We accomplish this with a single call to 'test_tick' at the start
of the test.
Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
link_alt_odb_entry: fix read over array bounds reported by valgrind
pfxlen can be longer than the path in objdir when relative_base
contains the path to gits object directory. Here we are interested
in checking if ent->base[] (the part that corresponds to .git/objects)
is the same string as objdir, and the code NUL-terminated ent->base[]
to
LEADING PATH\0XX/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\0
in preparation for these "duplicate check" step (before we return
from the function, the first NUL is turned into '/' so that we can
fill XX when probing for loose objects). All we need to do is to
compare the string with the path to our object directory.
Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Merge branch 'extract-remaining' of git://git.bogomips.org/git-svn
* 'extract-remaining' of git://git.bogomips.org/git-svn:
Extract Git::SVN::GlobSpec from git-svn.
Move Git::IndexInfo into its own file.
Load all the modules in one place and before running code.
Extract Git::SVN::Migration from git-svn.
Prepare Git::SVN::Migration for extraction from git-svn.
Extract Git::SVN::Log from git-svn.
Prepare Git::SVN::Log for extraction from git-svn.
* git://git.bogomips.org/git-svn:
Move initialization of Git::SVN variables into Git::SVN.
Extract Git::SVN from git-svn into its own .pm file.
Prepare Git::SVN for extraction into its own file.
Extract some utilities from git-svn to allow extracting Git::SVN.
perl: detect new files in MakeMaker builds
The Makefile.PL will now find .pm files itself.
Don't lose Error.pm if $@ gets clobbered.
Quiet warning if Makefile.PL is run with -w and no --localedir
Fix test breakages by a builder who does not have a valid user name
in his /etc/password entry.
* jk/autoident-test:
t7502: test early quit from commit with bad ident
t7502: handle systems where auto-identity is broken
t7502: drop confusing test_might_fail call
t7502: narrow checks for author/committer name in template
t7502: properly quote GIT_EDITOR
t7502: clean up fake_editor tests
In a superproject that has repository outside of its working tree,
"git submodule add" failed to clone a new submodule, as GIT_DIR and
GIT_WORK_TREE environment variables necessary to work in such a
superproject interfered with access to the submodule repository.
* dg/submodule-in-dismembered-working-tree:
git-submodule: work with GIT_DIR/GIT_WORK_TREE
"git checkout <branchname>" to come back from a detached HEAD state
incorrectly computed reachability of the detached HEAD, resulting in
unnecessary warnings.
* jk/maint-checkout-orphan-check-fix:
checkout: don't confuse ref and object flags