1#------------------------------------------------------------------------------ 2# Detect broken &&-chains in tests. 3# 4# At present, only &&-chains in subshells are examined by this linter; 5# top-level &&-chains are instead checked directly by the test framework. Like 6# the top-level &&-chain linter, the subshell linter (intentionally) does not 7# check &&-chains within {...} blocks. 8# 9# Checking for &&-chain breakage is done line-by-line by pure textual 10# inspection. 11# 12# Incomplete lines (those ending with "\") are stitched together with following 13# lines to simplify processing, particularly of "one-liner" statements. 14# Top-level here-docs are swallowed to avoid false positives within the 15# here-doc body, although the statement to which the here-doc is attached is 16# retained. 17# 18# Heuristics are used to detect end-of-subshell when the closing ")" is cuddled 19# with the final subshell statement on the same line: 20# 21# (cd foo && 22# bar) 23# 24# in order to avoid misinterpreting the ")" in constructs such as "x=$(...)" 25# and "case $x in *)" as ending the subshell. 26# 27# Lines missing a final "&&" are flagged with "?!AMP?!", and lines which chain 28# commands with ";" internally rather than "&&" are flagged "?!SEMI?!". A line 29# may be flagged for both violations. 30# 31# Detection of a missing &&-link in a multi-line subshell is complicated by the 32# fact that the last statement before the closing ")" must not end with "&&". 33# Since processing is line-by-line, it is not known whether a missing "&&" is 34# legitimate or not until the _next_ line is seen. To accommodate this, within 35# multi-line subshells, each line is stored in sed's "hold" area until after 36# the next line is seen and processed. If the next line is a stand-alone ")", 37# then a missing "&&" on the previous line is legitimate; otherwise a missing 38# "&&" is a break in the &&-chain. 39# 40# ( 41# cd foo && 42# bar 43# ) 44# 45# In practical terms, when "bar" is encountered, it is flagged with "?!AMP?!", 46# but when the stand-alone ")" line is seen which closes the subshell, the 47# "?!AMP?!" violation is removed from the "bar" line (retrieved from the "hold" 48# area) since the final statement of a subshell must not end with "&&". The 49# final line of a subshell may still break the &&-chain by using ";" internally 50# to chain commands together rather than "&&", so "?!SEMI?!" is never removed 51# from a line (even though "?!AMP?!" might be). 52# 53# Care is taken to recognize the last _statement_ of a multi-line subshell, not 54# necessarily the last textual _line_ within the subshell, since &&-chaining 55# applies to statements, not to lines. Consequently, blank lines, comment 56# lines, and here-docs are swallowed (but not the command to which the here-doc 57# is attached), leaving the last statement in the "hold" area, not the last 58# line, thus simplifying &&-link checking. 59# 60# The final statement before "done" in for- and while-loops, and before "elif", 61# "else", and "fi" in if-then-else likewise must not end with "&&", thus 62# receives similar treatment. 63# 64# Swallowing here-docs with arbitrary tags requires a bit of finesse. When a 65# line such as "cat <<EOF >out" is seen, the here-doc tag is moved to the front 66# of the line enclosed in angle brackets as a sentinel, giving "<EOF>cat >out". 67# As each subsequent line is read, it is appended to the target line and a 68# (whitespace-loose) back-reference match /^<(.*)>\n\1$/ is attempted to see if 69# the content inside "<...>" matches the entirety of the newly-read line. For 70# instance, if the next line read is "some data", when concatenated with the 71# target line, it becomes "<EOF>cat >out\nsome data", and a match is attempted 72# to see if "EOF" matches "some data". Since it doesn't, the next line is 73# attempted. When a line consisting of only "EOF" (and possible whitespace) is 74# encountered, it is appended to the target line giving "<EOF>cat >out\nEOF", 75# in which case the "EOF" inside "<...>" does match the text following the 76# newline, thus the closing here-doc tag has been found. The closing tag line 77# and the "<...>" prefix on the target line are then discarded, leaving just 78# the target line "cat >out". 79# 80# To facilitate regression testing (and manual debugging), a ">" annotation is 81# applied to the line containing ")" which closes a subshell, ">>" to a line 82# closing a nested subshell, and ">>>" to a line closing both at once. This 83# makes it easy to detect whether the heuristics correctly identify 84# end-of-subshell. 85#------------------------------------------------------------------------------ 86 87# incomplete line -- slurp up next line 88:squash 89/\\$/ { 90 N 91 s/\\\n// 92 bsquash 93} 94 95# here-doc -- swallow it to avoid false hits within its body (but keep the 96# command to which it was attached) 97/<<[ ]*[-\\']*[A-Za-z0-9_]/ { 98 s/^\(.*\)<<[ ]*[-\\']*\([A-Za-z0-9_][A-Za-z0-9_]*\)'*/<\2>\1<</ 99 s/[ ]*<<// 100 :hereslurp 101 N 102 /^<\([^>]*\)>.*\n[ ]*\1[ ]*$/!{ 103 s/\n.*$// 104 bhereslurp 105 } 106 s/^<[^>]*>// 107 s/\n.*$// 108} 109 110# one-liner "(...) &&" 111/^[ ]*!*[ ]*(..*)[ ]*&&[ ]*$/boneline 112 113# same as above but without trailing "&&" 114/^[ ]*!*[ ]*(..*)[ ]*$/boneline 115 116# one-liner "(...) >x" (or "2>x" or "<x" or "|x" or "&" 117/^[ ]*!*[ ]*(..*)[ ]*[0-9]*[<>|&]/boneline 118 119# multi-line "(...\n...)" 120/^[ ]*(/bsubshell 121 122# innocuous line -- print it and advance to next line 123b 124 125# found one-liner "(...)" -- mark suspect if it uses ";" internally rather than 126# "&&" (but not ";" in a string) 127:oneline 128/;/{ 129 /"[^"]*;[^"]*"/!s/^/?!SEMI?!/ 130} 131b 132 133:subshell 134# bare "(" line? 135/^[ ]*([ ]*$/ { 136 # stash for later printing 137 h 138 bnextline 139} 140# "(..." line -- split off and stash "(", then process "..." as its own line 141x 142s/.*/(/ 143x 144s/(// 145bslurp 146 147:nextline 148N 149s/.*\n// 150 151:slurp 152# incomplete line "...\" 153/\\$/bincomplete 154# multi-line quoted string "...\n..."? 155/"/bdqstring 156# multi-line quoted string '...\n...'? (but not contraction in string "it's") 157/'/{ 158 /"[^'"]*'[^'"]*"/!bsqstring 159} 160:folded 161# here-doc -- swallow it 162/<<[ ]*[-\\']*[A-Za-z0-9_]/bheredoc 163# comment or empty line -- discard since final non-comment, non-empty line 164# before closing ")", "done", "elsif", "else", or "fi" will need to be 165# re-visited to drop "suspect" marking since final line of those constructs 166# legitimately lacks "&&", so "suspect" mark must be removed 167/^[ ]*#/bnextline 168/^[ ]*$/bnextline 169# in-line comment -- strip it (but not "#" in a string, Bash ${#...} array 170# length, or Perforce "//depot/path#42" revision in filespec) 171/[ ]#/{ 172 /"[^"]*#[^"]*"/!s/[ ]#.*$// 173} 174# one-liner "case ... esac" 175/^[ ]*case[ ]*..*esac/bcheckchain 176# multi-line "case ... esac" 177/^[ ]*case[ ]..*[ ]in/bcase 178# multi-line "for ... done" or "while ... done" 179/^[ ]*for[ ]..*[ ]in/bcontinue 180/^[ ]*while[ ]/bcontinue 181/^[ ]*do[ ]/bcontinue 182/^[ ]*do[ ]*$/bcontinue 183/;[ ]*do/bcontinue 184/^[ ]*done[ ]*&&[ ]*$/bdone 185/^[ ]*done[ ]*$/bdone 186/^[ ]*done[ ]*[<>|]/bdone 187/^[ ]*done[ ]*)/bdone 188/||[ ]*exit[ ]/bcontinue 189/||[ ]*exit[ ]*$/bcontinue 190# multi-line "if...elsif...else...fi" 191/^[ ]*if[ ]/bcontinue 192/^[ ]*then[ ]/bcontinue 193/^[ ]*then[ ]*$/bcontinue 194/;[ ]*then/bcontinue 195/^[ ]*elif[ ]/belse 196/^[ ]*elif[ ]*$/belse 197/^[ ]*else[ ]/belse 198/^[ ]*else[ ]*$/belse 199/^[ ]*fi[ ]*&&[ ]*$/bdone 200/^[ ]*fi[ ]*$/bdone 201/^[ ]*fi[ ]*[<>|]/bdone 202/^[ ]*fi[ ]*)/bdone 203# nested one-liner "(...) &&" 204/^[ ]*(.*)[ ]*&&[ ]*$/bcheckchain 205# nested one-liner "(...)" 206/^[ ]*(.*)[ ]*$/bcheckchain 207# nested one-liner "(...) >x" (or "2>x" or "<x" or "|x") 208/^[ ]*(.*)[ ]*[0-9]*[<>|]/bcheckchain 209# nested multi-line "(...\n...)" 210/^[ ]*(/bnest 211# multi-line "{...\n...}" 212/^[ ]*{/bblock 213# closing ")" on own line -- exit subshell 214/^[ ]*)/bclosesolo 215# "$((...))" -- arithmetic expansion; not closing ")" 216/\$(([^)][^)]*))[^)]*$/bcheckchain 217# "$(...)" -- command substitution; not closing ")" 218/\$([^)][^)]*)[^)]*$/bcheckchain 219# multi-line "$(...\n...)" -- command substitution; treat as nested subshell 220/\$([^)]*$/bnest 221# "=(...)" -- Bash array assignment; not closing ")" 222/=(/bcheckchain 223# closing "...) &&" 224/)[ ]*&&[ ]*$/bclose 225# closing "...)" 226/)[ ]*$/bclose 227# closing "...) >x" (or "2>x" or "<x" or "|x") 228/)[ ]*[<>|]/bclose 229:checkchain 230# mark suspect if line uses ";" internally rather than "&&" (but not ";" in a 231# string and not ";;" in one-liner "case...esac") 232/;/{ 233 /;;/!{ 234 /"[^"]*;[^"]*"/!s/^/?!SEMI?!/ 235 } 236} 237# line ends with pipe "...|" -- valid; not missing "&&" 238/|[ ]*$/bcontinue 239# missing end-of-line "&&" -- mark suspect 240/&&[ ]*$/!s/^/?!AMP?!/ 241:continue 242# retrieve and print previous line 243x 244n 245bslurp 246 247# found incomplete line "...\" -- slurp up next line 248:incomplete 249N 250s/\\\n// 251bslurp 252 253# check for multi-line double-quoted string "...\n..." -- fold to one line 254:dqstring 255# remove all quote pairs 256s/"\([^"]*\)"/@!\1@!/g 257# done if no dangling quote 258/"/!bdqdone 259# otherwise, slurp next line and try again 260N 261s/\n// 262bdqstring 263:dqdone 264s/@!/"/g 265bfolded 266 267# check for multi-line single-quoted string '...\n...' -- fold to one line 268:sqstring 269# remove all quote pairs 270s/'\([^']*\)'/@!\1@!/g 271# done if no dangling quote 272/'/!bsqdone 273# otherwise, slurp next line and try again 274N 275s/\n// 276bsqstring 277:sqdone 278s/@!/'/g 279bfolded 280 281# found here-doc -- swallow it to avoid false hits within its body (but keep 282# the command to which it was attached) 283:heredoc 284s/^\(.*\)<<[ ]*[-\\']*\([A-Za-z0-9_][A-Za-z0-9_]*\)'*/<\2>\1<</ 285s/[ ]*<<// 286:hereslurpsub 287N 288/^<\([^>]*\)>.*\n[ ]*\1[ ]*$/!{ 289 s/\n.*$// 290 bhereslurpsub 291} 292s/^<[^>]*>// 293s/\n.*$// 294bfolded 295 296# found "case ... in" -- pass through untouched 297:case 298x 299n 300/^[ ]*esac/bslurp 301bcase 302 303# found "else" or "elif" -- drop "suspect" from final line before "else" since 304# that line legitimately lacks "&&" 305:else 306x 307s/?!AMP?!// 308x 309bcontinue 310 311# found "done" closing for-loop or while-loop, or "fi" closing if-then -- drop 312# "suspect" from final contained line since that line legitimately lacks "&&" 313:done 314x 315s/?!AMP?!// 316x 317# is 'done' or 'fi' cuddled with ")" to close subshell? 318/done.*)/bclose 319/fi.*)/bclose 320bcheckchain 321 322# found nested multi-line "(...\n...)" -- pass through untouched 323:nest 324x 325:nestslurp 326n 327# closing ")" on own line -- stop nested slurp 328/^[ ]*)/bnestclose 329# comment -- not closing ")" if in comment 330/^[ ]*#/bnestcontinue 331# "$((...))" -- arithmetic expansion; not closing ")" 332/\$(([^)][^)]*))[^)]*$/bnestcontinue 333# "$(...)" -- command substitution; not closing ")" 334/\$([^)][^)]*)[^)]*$/bnestcontinue 335# closing "...)" -- stop nested slurp 336/)/bnestclose 337:nestcontinue 338x 339bnestslurp 340:nestclose 341s/^/>>/ 342# is it "))" which closes nested and parent subshells? 343/)[ ]*)/bslurp 344bcheckchain 345 346# found multi-line "{...\n...}" block -- pass through untouched 347:block 348x 349n 350# closing "}" -- stop block slurp 351/}/bcheckchain 352bblock 353 354# found closing ")" on own line -- drop "suspect" from final line of subshell 355# since that line legitimately lacks "&&" and exit subshell loop 356:closesolo 357x 358s/?!AMP?!// 359p 360x 361s/^/>/ 362b 363 364# found closing "...)" -- exit subshell loop 365:close 366x 367p 368x 369s/^/>/ 370b