fsmonitor: add documentation for the fsmonitor extension.

This includes the core.fsmonitor setting, the fsmonitor integration hook,
and the fsmonitor index extension.

Also add documentation for the new fsmonitor options to ls-files and
update-index.

Signed-off-by: Ben Peart <benpeart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/Documentation/config.txt b/Documentation/config.txt
index dc4e3f58a2d2e8a5a6753bd290d65fec4f7f7b21..db52645cb4d1b0abe5e28af3357aac0590e127a1 100644 (file)
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -413,6 +413,13 @@ core.protectNTFS::
        8.3 "short" names.
        Defaults to `true` on Windows, and `false` elsewhere.
 
        8.3 "short" names.
        Defaults to `true` on Windows, and `false` elsewhere.
 

+core.fsmonitor::
+       If set, the value of this variable is used as a command which
+       will identify all files that may have changed since the
+       requested date/time. This information is used to speed up git by
+       avoiding unnecessary processing of files that have not changed.
+       See the "fsmonitor-watchman" section of linkgit:githooks[5].
+

 core.trustctime::
        If false, the ctime differences between the index and the
        working tree are ignored; useful when the inode change time
 core.trustctime::
        If false, the ctime differences between the index and the
        working tree are ignored; useful when the inode change time

diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt
index d153c17e0660d8ab9edb0b10a53172007d355510..3ac3e3a77d171d55a2992b263e03350cf4c08b3d 100644 (file)
--- a/Documentation/git-ls-files.txt
+++ b/Documentation/git-ls-files.txt
@@ -9,7 +9,7 @@ git-ls-files - Show information about files in the index and the working tree
 SYNOPSIS
 --------
 [verse]
 SYNOPSIS
 --------
 [verse]

-'git ls-files' [-z] [-t] [-v]
+'git ls-files' [-z] [-t] [-v] [-f]

                (--[cached|deleted|others|ignored|stage|unmerged|killed|modified])*
                (-[c|d|o|i|s|u|k|m])*
                [--eol]
                (--[cached|deleted|others|ignored|stage|unmerged|killed|modified])*
                (-[c|d|o|i|s|u|k|m])*
                [--eol]


@@ -133,6 +133,11 @@ a space) at the start of each line:
        that are marked as 'assume unchanged' (see
        linkgit:git-update-index[1]).
 
        that are marked as 'assume unchanged' (see
        linkgit:git-update-index[1]).
 

+-f::
+       Similar to `-t`, but use lowercase letters for files
+       that are marked as 'fsmonitor valid' (see
+       linkgit:git-update-index[1]).
+

 --full-name::
        When run from a subdirectory, the command usually
        outputs paths relative to the current directory.  This
 --full-name::
        When run from a subdirectory, the command usually
        outputs paths relative to the current directory.  This

diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt
index e19eba62cda292cc51407cbcd7e94675560f97f2..7c2f880a222e840c4c61cc57460e2b9a6a0a37a2 100644 (file)
--- a/Documentation/git-update-index.txt
+++ b/Documentation/git-update-index.txt
@@ -16,9 +16,11 @@ SYNOPSIS
             [--chmod=(+|-)x]
             [--[no-]assume-unchanged]
             [--[no-]skip-worktree]
             [--chmod=(+|-)x]
             [--[no-]assume-unchanged]
             [--[no-]skip-worktree]

+            [--[no-]fsmonitor-valid]

             [--ignore-submodules]
             [--[no-]split-index]
             [--[no-|test-|force-]untracked-cache]
             [--ignore-submodules]
             [--[no-]split-index]
             [--[no-|test-|force-]untracked-cache]

+            [--[no-]fsmonitor]

             [--really-refresh] [--unresolve] [--again | -g]
             [--info-only] [--index-info]
             [-z] [--stdin] [--index-version <n>]
             [--really-refresh] [--unresolve] [--again | -g]
             [--info-only] [--index-info]
             [-z] [--stdin] [--index-version <n>]


@@ -111,6 +113,12 @@ you will need to handle the situation manually.
        set and unset the "skip-worktree" bit for the paths. See
        section "Skip-worktree bit" below for more information.
 
        set and unset the "skip-worktree" bit for the paths. See
        section "Skip-worktree bit" below for more information.
 

+--[no-]fsmonitor-valid::
+       When one of these flags is specified, the object name recorded
+       for the paths are not updated. Instead, these options
+       set and unset the "fsmonitor valid" bit for the paths. See
+       section "File System Monitor" below for more information.
+

 -g::
 --again::
        Runs 'git update-index' itself on the paths whose index
 -g::
 --again::
        Runs 'git update-index' itself on the paths whose index


@@ -201,6 +209,15 @@ will remove the intended effect of the option.
        `--untracked-cache` used to imply `--test-untracked-cache` but
        this option would enable the extension unconditionally.
 
        `--untracked-cache` used to imply `--test-untracked-cache` but
        this option would enable the extension unconditionally.
 

+--fsmonitor::
+--no-fsmonitor::
+       Enable or disable files system monitor feature. These options
+       take effect whatever the value of the `core.fsmonitor`
+       configuration variable (see linkgit:git-config[1]). But a warning
+       is emitted when the change goes against the configured value, as
+       the configured value will take effect next time the index is
+       read and this will remove the intended effect of the option.
+

 \--::
        Do not interpret any more arguments as options.
 
 \--::
        Do not interpret any more arguments as options.
 


@@ -447,6 +464,34 @@ command reads the index; while when `--[no-|force-]untracked-cache`
 are used, the untracked cache is immediately added to or removed from
 the index.
 
 are used, the untracked cache is immediately added to or removed from
 the index.
 

+File System Monitor
+-------------------
+
+This feature is intended to speed up git operations for repos that have
+large working directories.
+
+It enables git to work together with a file system monitor (see the
+"fsmonitor-watchman" section of linkgit:githooks[5]) that can
+inform it as to what files have been modified. This enables git to avoid
+having to lstat() every file to find modified files.
+
+When used in conjunction with the untracked cache, it can further improve
+performance by avoiding the cost of scanning the entire working directory
+looking for new files.
+
+If you want to enable (or disable) this feature, it is easier to use
+the `core.fsmonitor` configuration variable (see
+linkgit:git-config[1]) than using the `--fsmonitor` option to
+`git update-index` in each repository, especially if you want to do so
+across all repositories you use, because you can set the configuration
+variable to `true` (or `false`) in your `$HOME/.gitconfig` just once
+and have it affect all repositories you touch.
+
+When the `core.fsmonitor` configuration variable is changed, the
+file system monitor is added to or removed from the index the next time
+a command reads the index. When `--[no-]fsmonitor` are used, the file
+system monitor is immediately added to or removed from the index.
+

 Configuration
 -------------
 
 Configuration
 -------------
 

diff --git a/Documentation/githooks.txt b/Documentation/githooks.txt
index 1bb4f92d4d317e16929767066a61c123fee0f280..ae60559cd91b35cb794200ea517842cb5fe74fdf 100644 (file)
--- a/Documentation/githooks.txt
+++ b/Documentation/githooks.txt
@@ -455,6 +455,34 @@ the name of the file that holds the e-mail to be sent.  Exiting with a
 non-zero status causes 'git send-email' to abort before sending any
 e-mails.
 
 non-zero status causes 'git send-email' to abort before sending any
 e-mails.
 

+fsmonitor-watchman
+~~~~~~~~~~~~~~~~~~
+
+This hook is invoked when the configuration option core.fsmonitor is
+set to .git/hooks/fsmonitor-watchman.  It takes two arguments, a version
+(currently 1) and the time in elapsed nanoseconds since midnight,
+January 1, 1970.
+
+The hook should output to stdout the list of all files in the working
+directory that may have changed since the requested time.  The logic
+should be inclusive so that it does not miss any potential changes.
+The paths should be relative to the root of the working directory
+and be separated by a single NUL.
+
+It is OK to include files which have not actually changed.  All changes
+including newly-created and deleted files should be included. When
+files are renamed, both the old and the new name should be included.
+
+Git will limit what files it checks for changes as well as which
+directories are checked for untracked files based on the path names
+given.
+
+An optimized way to tell git "all files have changed" is to return
+the filename '/'.
+
+The exit status determines whether git will use the data from the
+hook to limit its search.  On error, it will fall back to verifying
+all files and folders.

 
 GIT
 ---
 
 GIT
 ---

diff --git a/Documentation/technical/index-format.txt b/Documentation/technical/index-format.txt
index ade0b0c4454651812e88b292a846f23a58925f5f..db3572626beba6fc04f169b4ccccca0084d7326b 100644 (file)
--- a/Documentation/technical/index-format.txt
+++ b/Documentation/technical/index-format.txt
@@ -295,3 +295,22 @@ The remaining data of each directory block is grouped by type:
     in the previous ewah bitmap.
 
   - One NUL.
     in the previous ewah bitmap.
 
   - One NUL.

+
+== File System Monitor cache
+
+  The file system monitor cache tracks files for which the core.fsmonitor
+  hook has told us about changes.  The signature for this extension is
+  { 'F', 'S', 'M', 'N' }.
+
+  The extension starts with
+
+  - 32-bit version number: the current supported version is 1.
+
+  - 64-bit time: the extension data reflects all changes through the given
+       time which is stored as the nanoseconds elapsed since midnight,
+       January 1, 1970.
+
+  - 32-bit bitmap size: the size of the CE_FSMONITOR_VALID bitmap.
+
+  - An ewah bitmap, the n-th bit indicates whether the n-th index entry
+    is not CE_FSMONITOR_VALID.