logo
down
shadow

Asynchronous subprocess on Windows


Asynchronous subprocess on Windows

By : user3860727
Date : November 18 2020, 07:00 PM
wish helps you How bad is it to have to use threads? I encountered much the same problem and eventually decided to use threads to gather up all the data on a sub-process's stdout and stderr and put it onto a thread-safe queue which which the main thread can read in a blocking fashion, without having to worry about the threading going on behind the scenes.
It's not clear what trouble you anticipate with a solution based on threads and blocking. Are you worried about having to make the rest of your code thread-safe? That shouldn't be an issue since the IO thread wouldn't need to interact with any of the rest of your code or data. If you have very restrictive memory requirements or your pipeline is particularly long then perhaps you may feel unhappy about spawning so many threads. I don't know enough about your situation so I couldn't say if this is likely to be a problem, but it seems to me that since you're already spawning off extra processes a few threads to interact with them should not be a terrible burden. In my situation I have not found these IO threads to be particularly problematic.
code :


Share : facebook icon twitter icon
asynchronous subprocess with Popen

asynchronous subprocess with Popen


By : Yan Solopov
Date : March 29 2020, 07:55 AM
help you fix your problem Using Windows 7 + python 2.6, I am trying to run a simulation model in parallel. I can launch multiple instances of the executable by double-clicking on them in my file browser. However, asynchronous calls with Popen result in each successive instance interrupting the previous one. For what it's worth, the executable returns text to the console, but I don't need to collect results interactively. , Enabled by @J.F. Sebastian's comment regarding the cwd argument.
code :
import multiprocessing, subprocess

def run(c):
    exe = os.path.join("<location>","folder",str(c),"program.exe")
    run = os.path.join("<location>","folder",str(c),"run.dat")
    subprocess.check_call([exe,run],cwd=os.path.join("<location>","folder"),creationflags = subprocess.CREATE_NEW_CONSOLE)

def main():
    pool = multiprocessing.Pool(3)
    for c in range(10):
        pool.apply_async(run,(str(c),))
    pool.close()
    pool.join()

if __name__ == '__main__':
    main()
Subprocess.call or Subprocess.Popen cannot use executables that are in PATH (Linux/Windows)

Subprocess.call or Subprocess.Popen cannot use executables that are in PATH (Linux/Windows)


By : zozaf
Date : March 29 2020, 07:55 AM
it fixes the issue I'm writing a programme that needs to run on both Linux and Windows and use executables (with parameters) that exist in the path. (Assumed) , Ok here is how I got it to work.
code :
env = os.environ
proc = subprocess.Popen(args, env=env)
asynchronous subprocess with timeout

asynchronous subprocess with timeout


By : Jo9
Date : March 29 2020, 07:55 AM
seems to work fine As mentioned previously, the call to process.communicate() is making your code wait for the completion of the subprocess. However, if you just remove the communicate() call, the thread will exit immediately after spawning the process, causing your thread.join() call to exit too soon, and you'll kill off the subprocess prematurely. To do what you want without polling or busy waiting, you can set a timer that will kill the process (and runner thread) after a timeout if the process has not yet finished:
code :
class Command(object):
  def __init__(self, cmd):
    self.cmd = cmd
    self.process = None

  def run(self, timeout):
    def target():
      print('Thread started')
      # May want/need to skip the shlex.split() when using shell=True
      # See Popen() constructor docs on 'shell' argument for more detail.
      args = shlex.split(self.cmd)
      self.process = subprocess.Popen(args, shell=True)
      self.timer.start()
      self.process.wait()
      self.timer.cancel()

    def timer_callback():
        print('Terminating process (timed out)')
        self.process.terminate()

    thread = threading.Thread(target=target)
    self.timer = threading.Timer(timeout, timer_callback)
    thread.start()
Communicate with subprocess without waiting for the subprocess to terminate on windows

Communicate with subprocess without waiting for the subprocess to terminate on windows


By : user6217778
Date : March 29 2020, 07:55 AM
I wish did fix the issue. I have a simple echoprocess.py: , The main problem is with the line...
code :
print proc.stdout.read()
proc.stdin.write("hello\n")
print proc.stdout.readline()
data = sys.stdin.read()
data = sys.stdin.readline()
import sys

while True:
    data = sys.stdin.readline()
    sys.stdout.write("Here is the data: " + str(data))
    sys.stdout.flush()
from subprocess import Popen, PIPE

proc = Popen(["C:/python27/python.exe", "echoprocess.py"],
             stdin = PIPE,
             stdout = PIPE)

proc.stdin.write("hello\n")
proc.stdin.flush()
print proc.stdout.readline()
Running R script with subprocess on Windows, subprocess seems missing

Running R script with subprocess on Windows, subprocess seems missing


By : vamsi krishna
Date : March 29 2020, 07:55 AM
hope this fix your issue Check your python script and Rscript in are the same file location.
Example: Create a file named sb.R and subprocess_demo.py under Desktop location. Change directory to Desktop in command prompt and run following command python subprocess_demo.py
code :
c <- 3
print(paste("hello", c))
import subprocess
subprocess.call(['Rscript', "sb.R"])
Related Posts Related Posts :
  • Removing non-breaking spaces from strings using Python
  • Is there a way to backup everything in an app-engine blobstore?
  • Process a set of files from a source directory to a destination directory in Python
  • How to know if optparse option was passed in the command line or as a default
  • Rewriting a for loop in pure NumPy to decrease execution time
  • Comparing a time delta in python
  • Find next lower item in a sorted list
  • MySQLdb not INSERTING, _mysql does fine
  • Scipy Negative Distance? What?
  • What are the common patterns in web programming?
  • Percent signs in windows path
  • How to add a random number to a subsection of a numpy array?
  • How to generate all the values of an iterable besides the last few?
  • Searching by both class and range in XPath
  • Python code execution in Perl interpreter
  • Best Way to Include Variable in Python3
  • Serialize the @property methods in a Python class
  • What is the most platform- and Python-version-independent way to make a fast loop for use in Python?
  • Good way to edit the previous defined class in ipython
  • Bounced email on Google App Engine
  • Search jpeg files using python
  • Dynamically create class attributes
  • python unichr problem
  • Python beginner, strange output problem
  • Python: Finding a value in 1 list and finding that corresponding index in another list
  • can't install mysqlclient on mac os x mojave
  • Error indicates flattened dimensions when loading pre-trained network
  • how to replace underlines with words?
  • Adding through iteration
  • Use OpenCV on deployed Flask app (Heroku)
  • How to skip interstitial in a django view if a user hits the back button?
  • Any Naive Bayesian Classifier in python?
  • Python 2.5.2: remove what found between two lines that contain two concrete strings
  • Python 2.5.2 script that add "The function starts here" to all the functions of the files of a directory
  • HttpResponseRedirect question
  • Python socket error on UDP data receive. (10054)
  • Encoding issues with cloud ml
  • numpy.where - Weird behaviour: new elements spawning from nowhere?
  • I can't move my player in pygame, can you figure why?
  • Weird error I receive from Tkinter in Python
  • Using a Zapier Custom Request Webhook with JSON Web Tokens
  • Keras: Use categorical_crossentropy without one-hot encoded array of targets
  • Does python's httplib.HTTPConnection block?
  • Do alternate python implementation version numbers imply that they provide the same syntax?
  • Searching for specific HTML string using Python
  • python sax error "junk after document element"
  • MySql: How to know if an entry is compressed or not
  • Return a list of imported Python modules used in a script?
  • Returning a list in this recursive coi function in python
  • Python2.6 Decimal to Octal
  • Appengine Apps Vs Google bot web crawler
  • Changing models in django results in broken database?
  • Global variable functions
  • Using lambda inside a function
  • How to open a file, replace some strings, and save updates to the same file?
  • How to move the beginning of an input to the and?
  • If else fill variable if empty list
  • Pandas: Find and print all floats in column
  • sqlite3.OperationalError: database is locked - non-threaded application
  • How to implement mib module in net-snmp with python?
  • shadow
    Privacy Policy - Terms - Contact Us © 35dp-dentalpractice.co.uk