1# subprocess - Subprocesses with accessible I/O streams 2# 3# For more information about this module, see PEP 324. 4# 5# Copyright (c) 2003-2004 by Peter Astrand <astrand@lysator.liu.se> 6# 7# By obtaining, using, and/or copying this software and/or its 8# associated documentation, you agree that you have read, understood, 9# and will comply with the following terms and conditions: 10# 11# Permission to use, copy, modify, and distribute this software and 12# its associated documentation for any purpose and without fee is 13# hereby granted, provided that the above copyright notice appears in 14# all copies, and that both that copyright notice and this permission 15# notice appear in supporting documentation, and that the name of the 16# author not be used in advertising or publicity pertaining to 17# distribution of the software without specific, written prior 18# permission. 19# 20# THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, 21# INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. 22# IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, INDIRECT OR 23# CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS 24# OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, 25# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION 26# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. 27 28r"""subprocess - Subprocesses with accessible I/O streams 29 30This module allows you to spawn processes, connect to their 31input/output/error pipes, and obtain their return codes. This module 32intends to replace several other, older modules and functions, like: 33 34os.system 35os.spawn* 36os.popen* 37popen2.* 38commands.* 39 40Information about how the subprocess module can be used to replace these 41modules and functions can be found below. 42 43 44 45Using the subprocess module 46=========================== 47This module defines one class called Popen: 48 49class Popen(args, bufsize=0, executable=None, 50 stdin=None, stdout=None, stderr=None, 51 preexec_fn=None, close_fds=False, shell=False, 52 cwd=None, env=None, universal_newlines=False, 53 startupinfo=None, creationflags=0): 54 55 56Arguments are: 57 58args should be a string, or a sequence of program arguments. The 59program to execute is normally the first item in the args sequence or 60string, but can be explicitly set by using the executable argument. 61 62On UNIX, with shell=False (default): In this case, the Popen class 63uses os.execvp() to execute the child program. args should normally 64be a sequence. A string will be treated as a sequence with the string 65as the only item (the program to execute). 66 67On UNIX, with shell=True: If args is a string, it specifies the 68command string to execute through the shell. If args is a sequence, 69the first item specifies the command string, and any additional items 70will be treated as additional shell arguments. 71 72On Windows: the Popen class uses CreateProcess() to execute the child 73program, which operates on strings. If args is a sequence, it will be 74converted to a string using the list2cmdline method. Please note that 75not all MS Windows applications interpret the command line the same 76way: The list2cmdline is designed for applications using the same 77rules as the MS C runtime. 78 79bufsize, if given, has the same meaning as the corresponding argument 80to the built-in open() function: 0 means unbuffered, 1 means line 81buffered, any other positive value means use a buffer of 82(approximately) that size. A negative bufsize means to use the system 83default, which usually means fully buffered. The default value for 84bufsize is 0 (unbuffered). 85 86stdin, stdout and stderr specify the executed programs' standard 87input, standard output and standard error file handles, respectively. 88Valid values are PIPE, an existing file descriptor (a positive 89integer), an existing file object, and None. PIPE indicates that a 90new pipe to the child should be created. With None, no redirection 91will occur; the child's file handles will be inherited from the 92parent. Additionally, stderr can be STDOUT, which indicates that the 93stderr data from the applications should be captured into the same 94file handle as for stdout. 95 96If preexec_fn is set to a callable object, this object will be called 97in the child process just before the child is executed. 98 99If close_fds is true, all file descriptors except 0, 1 and 2 will be 100closed before the child process is executed. 101 102if shell is true, the specified command will be executed through the 103shell. 104 105If cwd is not None, the current directory will be changed to cwd 106before the child is executed. 107 108If env is not None, it defines the environment variables for the new 109process. 110 111If universal_newlines is true, the file objects stdout and stderr are 112opened as a text files, but lines may be terminated by any of '\n', 113the Unix end-of-line convention, '\r', the Macintosh convention or 114'\r\n', the Windows convention. All of these external representations 115are seen as '\n' by the Python program. Note: This feature is only 116available if Python is built with universal newline support (the 117default). Also, the newlines attribute of the file objects stdout, 118stdin and stderr are not updated by the communicate() method. 119 120The startupinfo and creationflags, if given, will be passed to the 121underlying CreateProcess() function. They can specify things such as 122appearance of the main window and priority for the new process. 123(Windows only) 124 125 126This module also defines two shortcut functions: 127 128call(*args, **kwargs): 129 Run command with arguments. Wait for command to complete, then 130 return the returncode attribute. 131 132 The arguments are the same as for the Popen constructor. Example: 133 134 retcode = call(["ls", "-l"]) 135 136 137Exceptions 138---------- 139Exceptions raised in the child process, before the new program has 140started to execute, will be re-raised in the parent. Additionally, 141the exception object will have one extra attribute called 142'child_traceback', which is a string containing traceback information 143from the childs point of view. 144 145The most common exception raised is OSError. This occurs, for 146example, when trying to execute a non-existent file. Applications 147should prepare for OSErrors. 148 149A ValueError will be raised if Popen is called with invalid arguments. 150 151 152Security 153-------- 154Unlike some other popen functions, this implementation will never call 155/bin/sh implicitly. This means that all characters, including shell 156metacharacters, can safely be passed to child processes. 157 158 159Popen objects 160============= 161Instances of the Popen class have the following methods: 162 163poll() 164 Check if child process has terminated. Returns returncode 165 attribute. 166 167wait() 168 Wait for child process to terminate. Returns returncode attribute. 169 170communicate(input=None) 171 Interact with process: Send data to stdin. Read data from stdout 172 and stderr, until end-of-file is reached. Wait for process to 173 terminate. The optional stdin argument should be a string to be 174 sent to the child process, or None, if no data should be sent to 175 the child. 176 177 communicate() returns a tuple (stdout, stderr). 178 179 Note: The data read is buffered in memory, so do not use this 180 method if the data size is large or unlimited. 181 182The following attributes are also available: 183 184stdin 185 If the stdin argument is PIPE, this attribute is a file object 186 that provides input to the child process. Otherwise, it is None. 187 188stdout 189 If the stdout argument is PIPE, this attribute is a file object 190 that provides output from the child process. Otherwise, it is 191 None. 192 193stderr 194 If the stderr argument is PIPE, this attribute is file object that 195 provides error output from the child process. Otherwise, it is 196 None. 197 198pid 199 The process ID of the child process. 200 201returncode 202 The child return code. A None value indicates that the process 203 hasn't terminated yet. A negative value -N indicates that the 204 child was terminated by signal N (UNIX only). 205 206 207Replacing older functions with the subprocess module 208==================================================== 209In this section, "a ==> b" means that b can be used as a replacement 210for a. 211 212Note: All functions in this section fail (more or less) silently if 213the executed program cannot be found; this module raises an OSError 214exception. 215 216In the following examples, we assume that the subprocess module is 217imported with "from subprocess import *". 218 219 220Replacing /bin/sh shell backquote 221--------------------------------- 222output=`mycmd myarg` 223==> 224output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0] 225 226 227Replacing shell pipe line 228------------------------- 229output=`dmesg | grep hda` 230==> 231p1 = Popen(["dmesg"], stdout=PIPE) 232p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) 233output = p2.communicate()[0] 234 235 236Replacing os.system() 237--------------------- 238sts = os.system("mycmd" + " myarg") 239==> 240p = Popen("mycmd" + " myarg", shell=True) 241sts = os.waitpid(p.pid, 0) 242 243Note: 244 245* Calling the program through the shell is usually not required. 246 247* It's easier to look at the returncode attribute than the 248 exitstatus. 249 250A more real-world example would look like this: 251 252try: 253 retcode = call("mycmd" + " myarg", shell=True) 254 if retcode < 0: 255 print >>sys.stderr, "Child was terminated by signal", -retcode 256 else: 257 print >>sys.stderr, "Child returned", retcode 258except OSError, e: 259 print >>sys.stderr, "Execution failed:", e 260 261 262Replacing os.spawn* 263------------------- 264P_NOWAIT example: 265 266pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg") 267==> 268pid = Popen(["/bin/mycmd", "myarg"]).pid 269 270 271P_WAIT example: 272 273retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg") 274==> 275retcode = call(["/bin/mycmd", "myarg"]) 276 277 278Vector example: 279 280os.spawnvp(os.P_NOWAIT, path, args) 281==> 282Popen([path] + args[1:]) 283 284 285Environment example: 286 287os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env) 288==> 289Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"}) 290 291 292Replacing os.popen* 293------------------- 294pipe = os.popen(cmd, mode='r', bufsize) 295==> 296pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout 297 298pipe = os.popen(cmd, mode='w', bufsize) 299==> 300pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin 301 302 303(child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize) 304==> 305p = Popen(cmd, shell=True, bufsize=bufsize, 306 stdin=PIPE, stdout=PIPE, close_fds=True) 307(child_stdin, child_stdout) = (p.stdin, p.stdout) 308 309 310(child_stdin, 311 child_stdout, 312 child_stderr) = os.popen3(cmd, mode, bufsize) 313==> 314p = Popen(cmd, shell=True, bufsize=bufsize, 315 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True) 316(child_stdin, 317 child_stdout, 318 child_stderr) = (p.stdin, p.stdout, p.stderr) 319 320 321(child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize) 322==> 323p = Popen(cmd, shell=True, bufsize=bufsize, 324 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True) 325(child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout) 326 327 328Replacing popen2.* 329------------------ 330Note: If the cmd argument to popen2 functions is a string, the command 331is executed through /bin/sh. If it is a list, the command is directly 332executed. 333 334(child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode) 335==> 336p = Popen(["somestring"], shell=True, bufsize=bufsize 337 stdin=PIPE, stdout=PIPE, close_fds=True) 338(child_stdout, child_stdin) = (p.stdout, p.stdin) 339 340 341(child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode) 342==> 343p = Popen(["mycmd", "myarg"], bufsize=bufsize, 344 stdin=PIPE, stdout=PIPE, close_fds=True) 345(child_stdout, child_stdin) = (p.stdout, p.stdin) 346 347The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen, 348except that: 349 350* subprocess.Popen raises an exception if the execution fails 351* the capturestderr argument is replaced with the stderr argument. 352* stdin=PIPE and stdout=PIPE must be specified. 353* popen2 closes all filedescriptors by default, but you have to specify 354 close_fds=True with subprocess.Popen. 355 356 357""" 358 359import sys 360mswindows = (sys.platform == "win32") 361 362import os 363import types 364import traceback 365 366if mswindows: 367 import threading 368 import msvcrt 369 if 0: # <-- change this to use pywin32 instead of the _subprocess driver 370 import pywintypes 371 from win32api import GetStdHandle, STD_INPUT_HANDLE, \ 372 STD_OUTPUT_HANDLE, STD_ERROR_HANDLE 373 from win32api import GetCurrentProcess, DuplicateHandle, \ 374 GetModuleFileName, GetVersion 375 from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE 376 from win32pipe import CreatePipe 377 from win32process import CreateProcess, STARTUPINFO, \ 378 GetExitCodeProcess, STARTF_USESTDHANDLES, \ 379 STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE 380 from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0 381 else: 382 from _subprocess import * 383 class STARTUPINFO: 384 dwFlags = 0 385 hStdInput = None 386 hStdOutput = None 387 hStdError = None 388 class pywintypes: 389 error = IOError 390else: 391 import select 392 import errno 393 import fcntl 394 import pickle 395 396__all__ = ["Popen", "PIPE", "STDOUT", "call"] 397 398try: 399 MAXFD = os.sysconf("SC_OPEN_MAX") 400except: 401 MAXFD = 256 402 403# True/False does not exist on 2.2.0 404try: 405 False 406except NameError: 407 False = 0 408 True = 1 409 410_active = [] 411 412def _cleanup(): 413 for inst in _active[:]: 414 inst.poll() 415 416PIPE = -1 417STDOUT = -2 418 419 420def call(*args, **kwargs): 421 """Run command with arguments. Wait for command to complete, then 422 return the returncode attribute. 423 424 The arguments are the same as for the Popen constructor. Example: 425 426 retcode = call(["ls", "-l"]) 427 """ 428 return Popen(*args, **kwargs).wait() 429 430 431def list2cmdline(seq): 432 """ 433 Translate a sequence of arguments into a command line 434 string, using the same rules as the MS C runtime: 435 436 1) Arguments are delimited by white space, which is either a 437 space or a tab. 438 439 2) A string surrounded by double quotation marks is 440 interpreted as a single argument, regardless of white space 441 contained within. A quoted string can be embedded in an 442 argument. 443 444 3) A double quotation mark preceded by a backslash is 445 interpreted as a literal double quotation mark. 446 447 4) Backslashes are interpreted literally, unless they 448 immediately precede a double quotation mark. 449 450 5) If backslashes immediately precede a double quotation mark, 451 every pair of backslashes is interpreted as a literal 452 backslash. If the number of backslashes is odd, the last 453 backslash escapes the next double quotation mark as 454 described in rule 3. 455 """ 456 457 # See 458 # http://msdn.microsoft.com/library/en-us/vccelng/htm/progs_12.asp 459 result = [] 460 needquote = False 461 for arg in seq: 462 bs_buf = [] 463 464 # Add a space to separate this argument from the others 465 if result: 466 result.append(' ') 467 468 needquote = (" " in arg) or ("\t" in arg) 469 if needquote: 470 result.append('"') 471 472 for c in arg: 473 if c == '\\': 474 # Don't know if we need to double yet. 475 bs_buf.append(c) 476 elif c == '"': 477 # Double backspaces. 478 result.append('\\' * len(bs_buf)*2) 479 bs_buf = [] 480 result.append('\\"') 481 else: 482 # Normal char 483 if bs_buf: 484 result.extend(bs_buf) 485 bs_buf = [] 486 result.append(c) 487 488 # Add remaining backspaces, if any. 489 if bs_buf: 490 result.extend(bs_buf) 491 492 if needquote: 493 result.extend(bs_buf) 494 result.append('"') 495 496 return ''.join(result) 497 498 499class Popen(object): 500 def __init__(self, args, bufsize=0, executable=None, 501 stdin=None, stdout=None, stderr=None, 502 preexec_fn=None, close_fds=False, shell=False, 503 cwd=None, env=None, universal_newlines=False, 504 startupinfo=None, creationflags=0): 505 """Create new Popen instance.""" 506 _cleanup() 507 508 if not isinstance(bufsize, (int, long)): 509 raise TypeError("bufsize must be an integer") 510 511 if mswindows: 512 if preexec_fn is not None: 513 raise ValueError("preexec_fn is not supported on Windows " 514 "platforms") 515 if close_fds: 516 raise ValueError("close_fds is not supported on Windows " 517 "platforms") 518 else: 519 # POSIX 520 if startupinfo is not None: 521 raise ValueError("startupinfo is only supported on Windows " 522 "platforms") 523 if creationflags != 0: 524 raise ValueError("creationflags is only supported on Windows " 525 "platforms") 526 527 self.stdin = None 528 self.stdout = None 529 self.stderr = None 530 self.pid = None 531 self.returncode = None 532 self.universal_newlines = universal_newlines 533 534 # Input and output objects. The general principle is like 535 # this: 536 # 537 # Parent Child 538 # ------ ----- 539 # p2cwrite ---stdin---> p2cread 540 # c2pread <--stdout--- c2pwrite 541 # errread <--stderr--- errwrite 542 # 543 # On POSIX, the child objects are file descriptors. On 544 # Windows, these are Windows file handles. The parent objects 545 # are file descriptors on both platforms. The parent objects 546 # are None when not using PIPEs. The child objects are None 547 # when not redirecting. 548 549 (p2cread, p2cwrite, 550 c2pread, c2pwrite, 551 errread, errwrite) = self._get_handles(stdin, stdout, stderr) 552 553 self._execute_child(args, executable, preexec_fn, close_fds, 554 cwd, env, universal_newlines, 555 startupinfo, creationflags, shell, 556 p2cread, p2cwrite, 557 c2pread, c2pwrite, 558 errread, errwrite) 559 560 if p2cwrite: 561 self.stdin = os.fdopen(p2cwrite, 'wb', bufsize) 562 if c2pread: 563 if universal_newlines: 564 self.stdout = os.fdopen(c2pread, 'rU', bufsize) 565 else: 566 self.stdout = os.fdopen(c2pread, 'rb', bufsize) 567 if errread: 568 if universal_newlines: 569 self.stderr = os.fdopen(errread, 'rU', bufsize) 570 else: 571 self.stderr = os.fdopen(errread, 'rb', bufsize) 572 573 _active.append(self) 574 575 576 def _translate_newlines(self, data): 577 data = data.replace("\r\n", "\n") 578 data = data.replace("\r", "\n") 579 return data 580 581 582 if mswindows: 583 # 584 # Windows methods 585 # 586 def _get_handles(self, stdin, stdout, stderr): 587 """Construct and return tupel with IO objects: 588 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 589 """ 590 if stdin == None and stdout == None and stderr == None: 591 return (None, None, None, None, None, None) 592 593 p2cread, p2cwrite = None, None 594 c2pread, c2pwrite = None, None 595 errread, errwrite = None, None 596 597 if stdin == None: 598 p2cread = GetStdHandle(STD_INPUT_HANDLE) 599 elif stdin == PIPE: 600 p2cread, p2cwrite = CreatePipe(None, 0) 601 # Detach and turn into fd 602 p2cwrite = p2cwrite.Detach() 603 p2cwrite = msvcrt.open_osfhandle(p2cwrite, 0) 604 elif type(stdin) == types.IntType: 605 p2cread = msvcrt.get_osfhandle(stdin) 606 else: 607 # Assuming file-like object 608 p2cread = msvcrt.get_osfhandle(stdin.fileno()) 609 p2cread = self._make_inheritable(p2cread) 610 611 if stdout == None: 612 c2pwrite = GetStdHandle(STD_OUTPUT_HANDLE) 613 elif stdout == PIPE: 614 c2pread, c2pwrite = CreatePipe(None, 0) 615 # Detach and turn into fd 616 c2pread = c2pread.Detach() 617 c2pread = msvcrt.open_osfhandle(c2pread, 0) 618 elif type(stdout) == types.IntType: 619 c2pwrite = msvcrt.get_osfhandle(stdout) 620 else: 621 # Assuming file-like object 622 c2pwrite = msvcrt.get_osfhandle(stdout.fileno()) 623 c2pwrite = self._make_inheritable(c2pwrite) 624 625 if stderr == None: 626 errwrite = GetStdHandle(STD_ERROR_HANDLE) 627 elif stderr == PIPE: 628 errread, errwrite = CreatePipe(None, 0) 629 # Detach and turn into fd 630 errread = errread.Detach() 631 errread = msvcrt.open_osfhandle(errread, 0) 632 elif stderr == STDOUT: 633 errwrite = c2pwrite 634 elif type(stderr) == types.IntType: 635 errwrite = msvcrt.get_osfhandle(stderr) 636 else: 637 # Assuming file-like object 638 errwrite = msvcrt.get_osfhandle(stderr.fileno()) 639 errwrite = self._make_inheritable(errwrite) 640 641 return (p2cread, p2cwrite, 642 c2pread, c2pwrite, 643 errread, errwrite) 644 645 646 def _make_inheritable(self, handle): 647 """Return a duplicate of handle, which is inheritable""" 648 return DuplicateHandle(GetCurrentProcess(), handle, 649 GetCurrentProcess(), 0, 1, 650 DUPLICATE_SAME_ACCESS) 651 652 653 def _find_w9xpopen(self): 654 """Find and return absolut path to w9xpopen.exe""" 655 w9xpopen = os.path.join(os.path.dirname(GetModuleFileName(0)), 656 "w9xpopen.exe") 657 if not os.path.exists(w9xpopen): 658 # Eeek - file-not-found - possibly an embedding 659 # situation - see if we can locate it in sys.exec_prefix 660 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix), 661 "w9xpopen.exe") 662 if not os.path.exists(w9xpopen): 663 raise RuntimeError("Cannot locate w9xpopen.exe, which is " 664 "needed for Popen to work with your " 665 "shell or platform.") 666 return w9xpopen 667 668 669 def _execute_child(self, args, executable, preexec_fn, close_fds, 670 cwd, env, universal_newlines, 671 startupinfo, creationflags, shell, 672 p2cread, p2cwrite, 673 c2pread, c2pwrite, 674 errread, errwrite): 675 """Execute program (MS Windows version)""" 676 677 if not isinstance(args, types.StringTypes): 678 args = list2cmdline(args) 679 680 # Process startup details 681 default_startupinfo = STARTUPINFO() 682 if startupinfo == None: 683 startupinfo = default_startupinfo 684 if not None in (p2cread, c2pwrite, errwrite): 685 startupinfo.dwFlags |= STARTF_USESTDHANDLES 686 startupinfo.hStdInput = p2cread 687 startupinfo.hStdOutput = c2pwrite 688 startupinfo.hStdError = errwrite 689 690 if shell: 691 default_startupinfo.dwFlags |= STARTF_USESHOWWINDOW 692 default_startupinfo.wShowWindow = SW_HIDE 693 comspec = os.environ.get("COMSPEC", "cmd.exe") 694 args = comspec + " /c " + args 695 if (GetVersion() >= 0x80000000L or 696 os.path.basename(comspec).lower() == "command.com"): 697 # Win9x, or using command.com on NT. We need to 698 # use the w9xpopen intermediate program. For more 699 # information, see KB Q150956 700 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp) 701 w9xpopen = self._find_w9xpopen() 702 args = '"%s" %s' % (w9xpopen, args) 703 # Not passing CREATE_NEW_CONSOLE has been known to 704 # cause random failures on win9x. Specifically a 705 # dialog: "Your program accessed mem currently in 706 # use at xxx" and a hopeful warning about the 707 # stability of your system. Cost is Ctrl+C wont 708 # kill children. 709 creationflags |= CREATE_NEW_CONSOLE 710 711 # Start the process 712 try: 713 hp, ht, pid, tid = CreateProcess(executable, args, 714 # no special security 715 None, None, 716 # must inherit handles to pass std 717 # handles 718 1, 719 creationflags, 720 env, 721 cwd, 722 startupinfo) 723 except pywintypes.error, e: 724 # Translate pywintypes.error to WindowsError, which is 725 # a subclass of OSError. FIXME: We should really 726 # translate errno using _sys_errlist (or simliar), but 727 # how can this be done from Python? 728 raise WindowsError(*e.args) 729 730 # Retain the process handle, but close the thread handle 731 self._handle = hp 732 self.pid = pid 733 ht.Close() 734 735 # Child is launched. Close the parent's copy of those pipe 736 # handles that only the child should have open. You need 737 # to make sure that no handles to the write end of the 738 # output pipe are maintained in this process or else the 739 # pipe will not close when the child process exits and the 740 # ReadFile will hang. 741 if p2cread != None: 742 p2cread.Close() 743 if c2pwrite != None: 744 c2pwrite.Close() 745 if errwrite != None: 746 errwrite.Close() 747 748 749 def poll(self): 750 """Check if child process has terminated. Returns returncode 751 attribute.""" 752 if self.returncode == None: 753 if WaitForSingleObject(self._handle, 0) == WAIT_OBJECT_0: 754 self.returncode = GetExitCodeProcess(self._handle) 755 _active.remove(self) 756 return self.returncode 757 758 759 def wait(self): 760 """Wait for child process to terminate. Returns returncode 761 attribute.""" 762 if self.returncode == None: 763 obj = WaitForSingleObject(self._handle, INFINITE) 764 self.returncode = GetExitCodeProcess(self._handle) 765 _active.remove(self) 766 return self.returncode 767 768 769 def _readerthread(self, fh, buffer): 770 buffer.append(fh.read()) 771 772 773 def communicate(self, input=None): 774 """Interact with process: Send data to stdin. Read data from 775 stdout and stderr, until end-of-file is reached. Wait for 776 process to terminate. The optional input argument should be a 777 string to be sent to the child process, or None, if no data 778 should be sent to the child. 779 780 communicate() returns a tuple (stdout, stderr).""" 781 stdout = None # Return 782 stderr = None # Return 783 784 if self.stdout: 785 stdout = [] 786 stdout_thread = threading.Thread(target=self._readerthread, 787 args=(self.stdout, stdout)) 788 stdout_thread.setDaemon(True) 789 stdout_thread.start() 790 if self.stderr: 791 stderr = [] 792 stderr_thread = threading.Thread(target=self._readerthread, 793 args=(self.stderr, stderr)) 794 stderr_thread.setDaemon(True) 795 stderr_thread.start() 796 797 if self.stdin: 798 if input != None: 799 self.stdin.write(input) 800 self.stdin.close() 801 802 if self.stdout: 803 stdout_thread.join() 804 if self.stderr: 805 stderr_thread.join() 806 807 # All data exchanged. Translate lists into strings. 808 if stdout != None: 809 stdout = stdout[0] 810 if stderr != None: 811 stderr = stderr[0] 812 813 # Translate newlines, if requested. We cannot let the file 814 # object do the translation: It is based on stdio, which is 815 # impossible to combine with select (unless forcing no 816 # buffering). 817 if self.universal_newlines and hasattr(open, 'newlines'): 818 if stdout: 819 stdout = self._translate_newlines(stdout) 820 if stderr: 821 stderr = self._translate_newlines(stderr) 822 823 self.wait() 824 return (stdout, stderr) 825 826 else: 827 # 828 # POSIX methods 829 # 830 def _get_handles(self, stdin, stdout, stderr): 831 """Construct and return tupel with IO objects: 832 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite 833 """ 834 p2cread, p2cwrite = None, None 835 c2pread, c2pwrite = None, None 836 errread, errwrite = None, None 837 838 if stdin == None: 839 pass 840 elif stdin == PIPE: 841 p2cread, p2cwrite = os.pipe() 842 elif type(stdin) == types.IntType: 843 p2cread = stdin 844 else: 845 # Assuming file-like object 846 p2cread = stdin.fileno() 847 848 if stdout == None: 849 pass 850 elif stdout == PIPE: 851 c2pread, c2pwrite = os.pipe() 852 elif type(stdout) == types.IntType: 853 c2pwrite = stdout 854 else: 855 # Assuming file-like object 856 c2pwrite = stdout.fileno() 857 858 if stderr == None: 859 pass 860 elif stderr == PIPE: 861 errread, errwrite = os.pipe() 862 elif stderr == STDOUT: 863 errwrite = c2pwrite 864 elif type(stderr) == types.IntType: 865 errwrite = stderr 866 else: 867 # Assuming file-like object 868 errwrite = stderr.fileno() 869 870 return (p2cread, p2cwrite, 871 c2pread, c2pwrite, 872 errread, errwrite) 873 874 875 def _set_cloexec_flag(self, fd): 876 try: 877 cloexec_flag = fcntl.FD_CLOEXEC 878 except AttributeError: 879 cloexec_flag = 1 880 881 old = fcntl.fcntl(fd, fcntl.F_GETFD) 882 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag) 883 884 885 def _close_fds(self, but): 886 for i in range(3, MAXFD): 887 if i == but: 888 continue 889 try: 890 os.close(i) 891 except: 892 pass 893 894 895 def _execute_child(self, args, executable, preexec_fn, close_fds, 896 cwd, env, universal_newlines, 897 startupinfo, creationflags, shell, 898 p2cread, p2cwrite, 899 c2pread, c2pwrite, 900 errread, errwrite): 901 """Execute program (POSIX version)""" 902 903 if isinstance(args, types.StringTypes): 904 args = [args] 905 906 if shell: 907 args = ["/bin/sh", "-c"] + args 908 909 if executable == None: 910 executable = args[0] 911 912 # For transferring possible exec failure from child to parent 913 # The first char specifies the exception type: 0 means 914 # OSError, 1 means some other error. 915 errpipe_read, errpipe_write = os.pipe() 916 self._set_cloexec_flag(errpipe_write) 917 918 self.pid = os.fork() 919 if self.pid == 0: 920 # Child 921 try: 922 # Close parent's pipe ends 923 if p2cwrite: 924 os.close(p2cwrite) 925 if c2pread: 926 os.close(c2pread) 927 if errread: 928 os.close(errread) 929 os.close(errpipe_read) 930 931 # Dup fds for child 932 if p2cread: 933 os.dup2(p2cread, 0) 934 if c2pwrite: 935 os.dup2(c2pwrite, 1) 936 if errwrite: 937 os.dup2(errwrite, 2) 938 939 # Close pipe fds. Make sure we doesn't close the same 940 # fd more than once. 941 if p2cread: 942 os.close(p2cread) 943 if c2pwrite and c2pwrite not in (p2cread,): 944 os.close(c2pwrite) 945 if errwrite and errwrite not in (p2cread, c2pwrite): 946 os.close(errwrite) 947 948 # Close all other fds, if asked for 949 if close_fds: 950 self._close_fds(but=errpipe_write) 951 952 if cwd != None: 953 os.chdir(cwd) 954 955 if preexec_fn: 956 apply(preexec_fn) 957 958 if env == None: 959 os.execvp(executable, args) 960 else: 961 os.execvpe(executable, args, env) 962 963 except: 964 exc_type, exc_value, tb = sys.exc_info() 965 # Save the traceback and attach it to the exception object 966 exc_lines = traceback.format_exception(exc_type, 967 exc_value, 968 tb) 969 exc_value.child_traceback = ''.join(exc_lines) 970 os.write(errpipe_write, pickle.dumps(exc_value)) 971 972 # This exitcode won't be reported to applications, so it 973 # really doesn't matter what we return. 974 os._exit(255) 975 976 # Parent 977 os.close(errpipe_write) 978 if p2cread and p2cwrite: 979 os.close(p2cread) 980 if c2pwrite and c2pread: 981 os.close(c2pwrite) 982 if errwrite and errread: 983 os.close(errwrite) 984 985 # Wait for exec to fail or succeed; possibly raising exception 986 data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB 987 os.close(errpipe_read) 988 if data != "": 989 os.waitpid(self.pid, 0) 990 child_exception = pickle.loads(data) 991 raise child_exception 992 993 994 def _handle_exitstatus(self, sts): 995 if os.WIFSIGNALED(sts): 996 self.returncode = -os.WTERMSIG(sts) 997 elif os.WIFEXITED(sts): 998 self.returncode = os.WEXITSTATUS(sts) 999 else:1000 # Should never happen1001 raise RuntimeError("Unknown child exit status!")10021003 _active.remove(self)100410051006 def poll(self):1007 """Check if child process has terminated. Returns returncode1008 attribute."""1009 if self.returncode == None:1010 try:1011 pid, sts = os.waitpid(self.pid, os.WNOHANG)1012 if pid == self.pid:1013 self._handle_exitstatus(sts)1014 except os.error:1015 pass1016 return self.returncode101710181019 def wait(self):1020 """Wait for child process to terminate. Returns returncode1021 attribute."""1022 if self.returncode == None:1023 pid, sts = os.waitpid(self.pid, 0)1024 self._handle_exitstatus(sts)1025 return self.returncode102610271028 def communicate(self, input=None):1029 """Interact with process: Send data to stdin. Read data from1030 stdout and stderr, until end-of-file is reached. Wait for1031 process to terminate. The optional input argument should be a1032 string to be sent to the child process, or None, if no data1033 should be sent to the child.10341035 communicate() returns a tuple (stdout, stderr)."""1036 read_set = []1037 write_set = []1038 stdout = None # Return1039 stderr = None # Return10401041 if self.stdin:1042 # Flush stdio buffer. This might block, if the user has1043 # been writing to .stdin in an uncontrolled fashion.1044 self.stdin.flush()1045 if input:1046 write_set.append(self.stdin)1047 else:1048 self.stdin.close()1049 if self.stdout:1050 read_set.append(self.stdout)1051 stdout = []1052 if self.stderr:1053 read_set.append(self.stderr)1054 stderr = []10551056 while read_set or write_set:1057 rlist, wlist, xlist = select.select(read_set, write_set, [])10581059 if self.stdin in wlist:1060 # When select has indicated that the file is writable,1061 # we can write up to PIPE_BUF bytes without risk1062 # blocking. POSIX defines PIPE_BUF >= 5121063 bytes_written = os.write(self.stdin.fileno(), input[:512])1064 input = input[bytes_written:]1065 if not input:1066 self.stdin.close()1067 write_set.remove(self.stdin)10681069 if self.stdout in rlist:1070 data = os.read(self.stdout.fileno(), 1024)1071 if data == "":1072 self.stdout.close()1073 read_set.remove(self.stdout)1074 stdout.append(data)10751076 if self.stderr in rlist:1077 data = os.read(self.stderr.fileno(), 1024)1078 if data == "":1079 self.stderr.close()1080 read_set.remove(self.stderr)1081 stderr.append(data)10821083 # All data exchanged. Translate lists into strings.1084 if stdout != None:1085 stdout = ''.join(stdout)1086 if stderr != None:1087 stderr = ''.join(stderr)10881089 # Translate newlines, if requested. We cannot let the file1090 # object do the translation: It is based on stdio, which is1091 # impossible to combine with select (unless forcing no1092 # buffering).1093 if self.universal_newlines and hasattr(open, 'newlines'):1094 if stdout:1095 stdout = self._translate_newlines(stdout)1096 if stderr:1097 stderr = self._translate_newlines(stderr)10981099 self.wait()1100 return (stdout, stderr)110111021103def _demo_posix():1104 #1105 # Example 1: Simple redirection: Get process list1106 #1107 plist = Popen(["ps"], stdout=PIPE).communicate()[0]1108 print "Process list:"1109 print plist11101111 #1112 # Example 2: Change uid before executing child1113 #1114 if os.getuid() == 0:1115 p = Popen(["id"], preexec_fn=lambda: os.setuid(100))1116 p.wait()11171118 #1119 # Example 3: Connecting several subprocesses1120 #1121 print "Looking for 'hda'..."1122 p1 = Popen(["dmesg"], stdout=PIPE)1123 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)1124 print repr(p2.communicate()[0])11251126 #1127 # Example 4: Catch execution error1128 #1129 print1130 print "Trying a weird file..."1131 try:1132 print Popen(["/this/path/does/not/exist"]).communicate()1133 except OSError, e:1134 if e.errno == errno.ENOENT:1135 print "The file didn't exist. I thought so..."1136 print "Child traceback:"1137 print e.child_traceback1138 else:1139 print "Error", e.errno1140 else:1141 print >>sys.stderr, "Gosh. No error."114211431144def _demo_windows():1145 #1146 # Example 1: Connecting several subprocesses1147 #1148 print "Looking for 'PROMPT' in set output..."1149 p1 = Popen("set", stdout=PIPE, shell=True)1150 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)1151 print repr(p2.communicate()[0])11521153 #1154 # Example 2: Simple execution of program1155 #1156 print "Executing calc..."1157 p = Popen("calc")1158 p.wait()115911601161if __name__ == "__main__":1162 if mswindows:1163 _demo_windows()1164 else:1165 _demo_posix()