shell进程的并行执行

有没有一个工具可以在 Windows 批处理文件中并行执行多个进程？我发现了一些有趣的 Linux 工具（parallel http://mi.eng.cam.ac.uk/~er258/code/parallel.html and PPSS http://code.google.com/p/ppss/），但是，我需要一个适用于 Windows 平台的工具。

奖励：如果该工具还允许在多台计算机之间以简单的方式分发进程，并远程运行进程，那就太好了PsExec http://technet.microsoft.com/de-de/sysinternals/bb897553.aspx.

示例：我希望在下面的 for 循环中

for %F in (*.*) do processFile.exe %F

有限数量的 processFile.exe 实例并行运行以利用多核 CPU。

Edit - 我修改了脚本以选择性地显示每个进程的输出

这是一个本机批处理解决方案，可以可靠地并行运行命令列表，永远不会启动超过n一次处理。

它甚至有一个内置机制，可以通过 PSEXEC 将进程分发到特定的 CPU 或远程计算机，但我尚未测试该功能。

完成这项工作的技巧是通过 CMD 进程启动每个命令，该进程将 stdout 或未定义的句柄重定向到锁定文件。该进程将保持文件上的独占锁，直到其终止。无论进程如何终止（正常退出、崩溃、进程被杀死），锁都会立即释放。

主脚本可以通过尝试重定向到同一锁定文件来测试进程是否仍处于活动状态。如果进程仍处于活动状态，则重定向将失败；如果进程已终止，则重定向将成功。

默认情况下，脚本忽略每个进程的输出。如果开始于/O选项作为第一个参数，然后它显示每个进程的输出，没有交错。

我的演示将进程限制设置为 4，并简单地运行一系列不同长度的 PING 命令。

我已经在 XP、Vista 和 Windows 7 上对此进行了测试。

@echo off
setlocal enableDelayedExpansion

:: Display the output of each process if the /O option is used
:: else ignore the output of each process
if /i "%~1" equ "/O" (
  set "lockHandle=1"
  set "showOutput=1"
) else (
  set "lockHandle=1^>nul 9"
  set "showOutput="
)

:: The list of commands could come from anywhere such as another file
:: or the output of another command. For this demo I will list the
:: commands within this script - Each command is prefixed with :::
::: ping /n 05 ::1
::: ping /n 20 ::1
::: ping /n 10 ::1
::: ping /n 15 ::1
::: ping /n 07 ::1
::: ping /n 05 ::1
::: ping /n 20 ::1
::: ping /n 10 ::1
::: ping /n 15 ::1
::: ping /n 07 ::1

:: Define the maximum number of parallel processes to run.
:: Each process number can optionally be assigned to a particular server
:: and/or cpu via psexec specs (untested).
set "maxProc=4"

:: Optional - Define CPU targets in terms of PSEXEC specs
::           (everything but the command)
::
:: If a CPU is not defined for a proc, then it will be run on the local machine.
:: I haven't tested this feature, but it seems like it should work.
::
:: set cpu1=psexec \\server1 ...
:: set cpu2=psexec \\server1 ...
:: set cpu3=psexec \\server2 ...
:: etc.

:: For this demo force all CPU specs to undefined (local machine)
for /l %%N in (1 1 %maxProc%) do set "cpu%%N="

:: Get a unique base lock name for this particular instantiation.
:: Incorporate a timestamp from WMIC if possible, but don't fail if
:: WMIC not available. Also incorporate a random number.
  set "lock="
  for /f "skip=1 delims=-+ " %%T in ('2^>nul wmic os get localdatetime') do (
    set "lock=%%T"
    goto :break
  )
  :break
  set "lock=%temp%\lock%lock%_%random%_"

:: Initialize the counters
  set /a "startCount=0, endCount=0"

:: Clear any existing end flags
  for /l %%N in (1 1 %maxProc%) do set "endProc%%N="

:: Launch the commands in a loop
:: Modify the IN () clause as needed to retrieve the list of commands
  set launch=1
  for /f "tokens=* delims=:" %%A in ('findstr /b ":::" "%~f0"') do (
    if !startCount! lss %maxProc% (
      set /a "startCount+=1, nextProc=startCount"
    ) else (
      call :wait
    )
    set cmd!nextProc!=%%A
    if defined showOutput echo -------------------------------------------------------------------------------
    echo !time! - proc!nextProc!: starting %%A
    2>nul del %lock%!nextProc!
    %= Redirect the lock handle to the lock file. The CMD process will     =%
    %= maintain an exclusive lock on the lock file until the process ends. =%
    start /b "" cmd /c %lockHandle%^>"%lock%!nextProc!" 2^>^&1 !cpu%%N! %%A
  )
  set "launch="

:wait
:: Wait for procs to finish in a loop
:: If still launching then return as soon as a proc ends
:: else wait for all procs to finish
  :: redirect stderr to null to suppress any error message if redirection
  :: within the loop fails.
  for /l %%N in (1 1 %startCount%) do 2>nul (
    %= Redirect an unused file handle to the lock file. If the process is    =%
    %= still running then redirection will fail and the IF body will not run =%
    if not defined endProc%%N if exist "%lock%%%N" 9>>"%lock%%%N" (
      %= Made it inside the IF body so the process must have finished =%
      if defined showOutput echo ===============================================================================
      echo !time! - proc%%N: finished !cmd%%N!
      if defined showOutput type "%lock%%%N"
      if defined launch (
        set nextProc=%%N
        exit /b
      )
      set /a "endCount+=1, endProc%%N=1"
    )
  )
  if %endCount% lss %startCount% (
    1>nul 2>nul ping /n 2 ::1
    goto :wait
  )

2>nul del %lock%*
if defined showOutput echo ===============================================================================
echo Thats all folks^^!

以下是忽略进程输出的示例运行的输出

12:24:07.52 - proc1: starting  ping /n 05 ::1
12:24:07.52 - proc2: starting  ping /n 20 ::1
12:24:07.53 - proc3: starting  ping /n 10 ::1
12:24:07.54 - proc4: starting  ping /n 15 ::1
12:24:11.60 - proc1: finished  ping /n 05 ::1
12:24:11.60 - proc1: starting  ping /n 07 ::1
12:24:16.66 - proc3: finished  ping /n 10 ::1
12:24:16.66 - proc3: starting  ping /n 05 ::1
12:24:17.68 - proc1: finished  ping /n 07 ::1
12:24:17.68 - proc1: starting  ping /n 20 ::1
12:24:20.72 - proc3: finished  ping /n 05 ::1
12:24:20.72 - proc3: starting  ping /n 10 ::1
12:24:21.75 - proc4: finished  ping /n 15 ::1
12:24:21.75 - proc4: starting  ping /n 15 ::1
12:24:26.82 - proc2: finished  ping /n 20 ::1
12:24:26.82 - proc2: starting  ping /n 07 ::1
12:24:29.86 - proc3: finished  ping /n 10 ::1
12:24:32.89 - proc2: finished  ping /n 07 ::1
12:24:35.92 - proc4: finished  ping /n 15 ::1
12:24:36.93 - proc1: finished  ping /n 20 ::1
Thats all folks!

如果使用以下命令运行，则输出如下/O显示进程输出的选项

-------------------------------------------------------------------------------
12:24:51.02 - proc1: starting  ping /n 05 ::1
-------------------------------------------------------------------------------
12:24:51.02 - proc2: starting  ping /n 20 ::1
-------------------------------------------------------------------------------
12:24:51.03 - proc3: starting  ping /n 10 ::1
-------------------------------------------------------------------------------
12:24:51.04 - proc4: starting  ping /n 15 ::1
===============================================================================
12:24:55.10 - proc1: finished  ping /n 05 ::1

Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Ping statistics for ::1:
    Packets: Sent = 5, Received = 5, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:24:55.10 - proc1: starting  ping /n 07 ::1
===============================================================================
12:25:00.17 - proc3: finished  ping /n 10 ::1

Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Ping statistics for ::1:
    Packets: Sent = 10, Received = 10, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:25:00.19 - proc3: starting  ping /n 05 ::1
===============================================================================
12:25:01.22 - proc1: finished  ping /n 07 ::1

Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Ping statistics for ::1:
    Packets: Sent = 7, Received = 7, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:25:01.23 - proc1: starting  ping /n 20 ::1
===============================================================================
12:25:04.27 - proc3: finished  ping /n 05 ::1

Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Ping statistics for ::1:
    Packets: Sent = 5, Received = 5, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:25:04.28 - proc3: starting  ping /n 10 ::1
===============================================================================
12:25:05.30 - proc4: finished  ping /n 15 ::1

Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Ping statistics for ::1:
    Packets: Sent = 15, Received = 15, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:25:05.32 - proc4: starting  ping /n 15 ::1
===============================================================================
12:25:10.38 - proc2: finished  ping /n 20 ::1

Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Ping statistics for ::1:
    Packets: Sent = 20, Received = 20, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
-------------------------------------------------------------------------------
12:25:10.40 - proc2: starting  ping /n 07 ::1
===============================================================================
12:25:13.44 - proc3: finished  ping /n 10 ::1

Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Ping statistics for ::1:
    Packets: Sent = 10, Received = 10, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
===============================================================================
12:25:16.48 - proc2: finished  ping /n 07 ::1

Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Ping statistics for ::1:
    Packets: Sent = 7, Received = 7, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
===============================================================================
12:25:19.52 - proc4: finished  ping /n 15 ::1

Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Ping statistics for ::1:
    Packets: Sent = 15, Received = 15, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
===============================================================================
12:25:20.54 - proc1: finished  ping /n 20 ::1

Pinging ::1 with 32 bytes of data:
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms
Reply from ::1: time<1ms

Ping statistics for ::1:
    Packets: Sent = 20, Received = 20, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
===============================================================================
Thats all folks!