gitweb: Load checking

This changes slightly the behavior of gitweb, so that it verifies
that the box isn't inundated with before attempting to serve gitweb.
If the box is overloaded, it basically returns a 503 Server Unavailable
until the load falls below the defined threshold.  This helps dramatically
if you have a box that's I/O bound, reaches a certain load and you
don't want gitweb, the I/O hog that it is, increasing the pain the
server is already undergoing.

This behavior is controlled by $maxload configuration variable.
Default is a load of 300, which for most cases should never be hit.
Unset it (set it to undefined value, i.e. undef) to turn off checking.

Currently it requires that '/proc/loadavg' file exists, otherwise the
load check is bypassed (load is taken to be 0).  So platforms that do
not implement '/proc/loadavg' currently cannot use this feature
(provisions are included for additional checks to be added by others).

There is simple test in t/t9501-gitweb-standalone-http-status.sh to
check that it correctly returns "503 Service Unavailable" if load is
too high, and also if there are any Perl warnings or errors.

Signed-off-by: John 'Warthog9' Hawley <warthog9@kernel.org>
Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/gitweb/README b/gitweb/README
index e34ee793ef51584344f8c808454b9ead0c3348c6..6c2c8e12598ca9ba17e7cc3d74812a8b7883d59a 100644 (file)
--- a/gitweb/README
+++ b/gitweb/README
@@ -174,7 +174,7 @@ not include variables usually directly set during build):
    Base URL for relative URLs in pages generated by gitweb,
    (e.g. $logo, $favicon, @stylesheets if they are relative URLs),
    needed and used only for URLs with nonempty PATH_INFO via
    Base URL for relative URLs in pages generated by gitweb,
    (e.g. $logo, $favicon, @stylesheets if they are relative URLs),
    needed and used only for URLs with nonempty PATH_INFO via

-   <base href="$base_url>.  Usually gitweb sets its value correctly,
+   <base href="$base_url">.  Usually gitweb sets its value correctly,

    and there is no need to set this variable, e.g. to $my_uri or "/".
  * $home_link
    Target of the home link on top of all pages (the first part of view
    and there is no need to set this variable, e.g. to $my_uri or "/".
  * $home_link
    Target of the home link on top of all pages (the first part of view


@@ -228,6 +228,11 @@ not include variables usually directly set during build):
    repositories from launching cross-site scripting (XSS) attacks.  Set this
    to true if you don't trust the content of your repositories. The default
    is false.
    repositories from launching cross-site scripting (XSS) attacks.  Set this
    to true if you don't trust the content of your repositories. The default
    is false.

+ * $maxload
+   Used to set the maximum load that we will still respond to gitweb queries.
+   If server load exceed this value then return "503 Service Unavaliable" error.
+   Server load is taken to be 0 if gitweb cannot determine its value.  Set it to
+   undefined value to turn it off.  The default is 300.

 
 
 Projects list file format
 
 
 Projects list file format

diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 7e477af9567cff322d18fc9a1fbfe507732418dc..e2522cc64f50fbf2a740dd7656373a99f9769119 100755 (executable)
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -221,6 +221,12 @@ BEGIN
        'double'  => 32
 );
 
        'double'  => 32
 );
 

+# Used to set the maximum load that we will still respond to gitweb queries.
+# If server load exceed this value then return "503 server busy" error.
+# If gitweb cannot determined server load, it is taken to be 0.
+# Leave it undefined (or set to 'undef') to turn off load checking.
+our $maxload = 300;
+

 # You define site-wide feature defaults here; override them with
 # $GITWEB_CONFIG as necessary.
 our %feature = (
 # You define site-wide feature defaults here; override them with
 # $GITWEB_CONFIG as necessary.
 our %feature = (


@@ -551,12 +557,38 @@ sub filter_snapshot_fmts {
        do $GITWEB_CONFIG_SYSTEM if -e $GITWEB_CONFIG_SYSTEM;
 }
 
        do $GITWEB_CONFIG_SYSTEM if -e $GITWEB_CONFIG_SYSTEM;
 }
 

+# Get loadavg of system, to compare against $maxload.
+# Currently it requires '/proc/loadavg' present to get loadavg;
+# if it is not present it returns 0, which means no load checking.
+sub get_loadavg {
+       if( -e '/proc/loadavg' ){
+               open my $fd, '<', '/proc/loadavg'
+                       or return 0;
+               my @load = split(/\s+/, scalar <$fd>);
+               close $fd;
+
+               # The first three columns measure CPU and IO utilization of the last one,
+               # five, and 10 minute periods.  The fourth column shows the number of
+               # currently running processes and the total number of processes in the m/n
+               # format.  The last column displays the last process ID used.
+               return $load[0] || 0;
+       }
+       # additional checks for load average should go here for things that don't export
+       # /proc/loadavg
+
+       return 0;
+}
+

 # version of the core git binary
 our $git_version = qx("$GIT" --version) =~ m/git version (.*)$/ ? $1 : "unknown";
 $number_of_git_cmds++;
 
 $projects_list ||= $projectroot;
 
 # version of the core git binary
 our $git_version = qx("$GIT" --version) =~ m/git version (.*)$/ ? $1 : "unknown";
 $number_of_git_cmds++;
 
 $projects_list ||= $projectroot;
 

+if (defined $maxload && get_loadavg() > $maxload) {
+       die_error(503, "The load average on the server is too high");
+}
+

 # ======================================================================
 # input validation and dispatch
 
 # ======================================================================
 # input validation and dispatch
 


@@ -3328,7 +3360,8 @@ sub git_footer_html {
        }
 
        print qq!<script type="text/javascript" src="$javascript"></script>\n!;
        }
 
        print qq!<script type="text/javascript" src="$javascript"></script>\n!;

-       if ($action eq 'blame_incremental') {
+       if (defined $action &&
+           $action eq 'blame_incremental') {

                print qq!<script type="text/javascript">\n!.
                      qq!startBlame("!. href(action=>"blame_data", -replay=>1) .qq!",\n!.
                      qq!           "!. href() .qq!");\n!.
                print qq!<script type="text/javascript">\n!.
                      qq!startBlame("!. href(action=>"blame_data", -replay=>1) .qq!",\n!.
                      qq!           "!. href() .qq!");\n!.


@@ -3354,14 +3387,19 @@ sub git_footer_html {
 # 500: The server isn't configured properly, or
 #      an internal error occurred (e.g. failed assertions caused by bugs), or
 #      an unknown error occurred (e.g. the git binary died unexpectedly).
 # 500: The server isn't configured properly, or
 #      an internal error occurred (e.g. failed assertions caused by bugs), or
 #      an unknown error occurred (e.g. the git binary died unexpectedly).

+# 503: The server is currently unavailable (because it is overloaded,
+#      or down for maintenance).  Generally, this is a temporary state.

 sub die_error {
        my $status = shift || 500;
        my $error = shift || "Internal server error";
 
 sub die_error {
        my $status = shift || 500;
        my $error = shift || "Internal server error";
 

-       my %http_responses = (400 => '400 Bad Request',
-                             403 => '403 Forbidden',
-                             404 => '404 Not Found',
-                             500 => '500 Internal Server Error');
+       my %http_responses = (
+               400 => '400 Bad Request',
+               403 => '403 Forbidden',
+               404 => '404 Not Found',
+               500 => '500 Internal Server Error',
+               503 => '503 Service Unavailable',
+       );

        git_header_html($http_responses{$status});
        print <<EOF;
 <div class="page_body">
        git_header_html($http_responses{$status});
        print <<EOF;
 <div class="page_body">

diff --git a/t/gitweb-lib.sh b/t/gitweb-lib.sh
index 76d8b7b803b4134b65f2d4f624e29223dda2a7ee..5a734b1b7b2df4a5c5c35f5347c618f3735317ac 100644 (file)
--- a/t/gitweb-lib.sh
+++ b/t/gitweb-lib.sh
@@ -25,6 +25,7 @@ our \$favicon = 'file:///$TEST_DIRECTORY/../gitweb/git-favicon.png';
 our \$projects_list = '';
 our \$export_ok = '';
 our \$strict_export = '';
 our \$projects_list = '';
 our \$export_ok = '';
 our \$strict_export = '';

+our \$maxload = undef;

 
 EOF
 
 
 EOF
 

diff --git a/t/t9501-gitweb-standalone-http-status.sh b/t/t9501-gitweb-standalone-http-status.sh
index 9e8bc01ea6e96a9263c31fd6563416727075a072..7590f10b66b6fafc5baf3f9751b57469248f8e83 100755 (executable)
--- a/t/t9501-gitweb-standalone-http-status.sh
+++ b/t/t9501-gitweb-standalone-http-status.sh
@@ -112,4 +112,26 @@ test_expect_success 'snapshots: bad object id' '
 test_debug 'cat gitweb.output'
 
 
 test_debug 'cat gitweb.output'
 
 

+# ----------------------------------------------------------------------
+# load checking
+
+# always hit the load limit
+cat >>gitweb_config.perl <<\EOF
+our $maxload = 0;
+EOF
+
+test_expect_success 'load checking: load too high (default action)' '
+       gitweb_run "p=.git" &&
+       grep "Status: 503 Service Unavailable" gitweb.headers &&
+       grep "503 - The load average on the server is too high" gitweb.body
+'
+test_debug 'cat gitweb.log' # just in case
+test_debug 'cat gitweb.headers'
+
+# turn off load checking
+cat >>gitweb_config.perl <<\EOF
+our $maxload = undef;
+EOF
+
+

 test_done
 test_done