在python多处理中修改对象

我有大量自定义对象，需要对其执行独立（可并行）任务，包括修改对象参数。我尝试过使用 Manager().dict 和“sharedmem”ory，但两者都不起作用。例如：

import numpy as np
import multiprocessing as mp
import sharedmem as shm


class Tester:

    num = 0.0
    name = 'none'
    def __init__(self,tnum=num, tname=name):
        self.num  = tnum
        self.name = tname

    def __str__(self):
        return '%f %s' % (self.num, self.name)

def mod(test, nn):
    test.num = np.random.randn()
    test.name = nn


if __name__ == '__main__':

    num = 10

    tests = np.empty(num, dtype=object)
    for it in range(num):
        tests[it] = Tester(tnum=it*1.0)

    sh_tests = shm.empty(num, dtype=object)
    for it in range(num):
        sh_tests[it] = tests[it]
        print sh_tests[it]

    print '\n'
    workers = [ mp.Process(target=mod, args=(test, 'some') ) for test in sh_tests ]

    for work in workers: work.start()

    for work in workers: work.join()

    for test in sh_tests: print test

打印出：

0.000000 none
1.000000 none
2.000000 none
3.000000 none
4.000000 none
5.000000 none
6.000000 none
7.000000 none
8.000000 none
9.000000 none


0.000000 none
1.000000 none
2.000000 none
3.000000 none
4.000000 none
5.000000 none
6.000000 none
7.000000 none
8.000000 none
9.000000 none

IE。对象没有被修改。

我怎样才能实现预期的行为？

问题是，当对象传递到工作进程时，它们会用 pickle 打包，运送到另一个进程，在那里它们被解包并进行处理。您的对象与其说是传递给其他进程，不如说是克隆。您不会返回对象，因此克隆的对象会被愉快地修改，然后被丢弃。

看来这不能完成（可以在两个独立的进程之间共享内存数据吗？ https://stackoverflow.com/questions/1268252/python-possible-to-share-in-memory-data-between-2-separate-processes/1269055#1269055） 直接地。

你能做的就是返回修改后的对象。

import numpy as np
import multiprocessing as mp


class Tester:

    num = 0.0
    name = 'none'
    def __init__(self,tnum=num, tname=name):
        self.num  = tnum
        self.name = tname

    def __str__(self):
        return '%f %s' % (self.num, self.name)

def mod(test, nn, out_queue):
    print test.num
    test.num = np.random.randn()
    print test.num
    test.name = nn
    out_queue.put(test)


if __name__ == '__main__':       
    num = 10
    out_queue = mp.Queue()
    tests = np.empty(num, dtype=object)
    for it in range(num):
        tests[it] = Tester(tnum=it*1.0)
    
    
    print '\n'
    workers = [ mp.Process(target=mod, args=(test, 'some', out_queue) ) for test in tests ]
    
    for work in workers: work.start()
    
    for work in workers: work.join()
    
    res_lst = []
    for j in range(len(workers)):
        res_lst.append(out_queue.get())
    
    for test in res_lst: print test

这确实导致了一个有趣的观察，因为产生的进程是相同的，它们都以相同的随机数种子开始，所以它们all生成相同的“随机”数字：

0.0
-0.824934300553
1.0
-0.824934300553
2.0
-0.824934300553
3.0
-0.824934300553
4.0
-0.824934300553
5.0
-0.824934300553
6.0
-0.824934300553
7.0
-0.824934300553
8.0
-0.824934300553
9.0
-0.824934300553
-0.824934 some
-0.824934 some
-0.824934 some
-0.824934 some
-0.824934 some
-0.824934 some
-0.824934 some
-0.824934 some
-0.824934 some
-0.824934 some