AWS Elastic mapreduce 似乎没有正确地将流媒体转换为 jar

2024-01-08

我有一个映射器和减速器,当我在管道版本中运行它们时,它们可以正常工作:

cat data.csv | ./mapper.py | sort -k1,1 | ./reducer.py

我使用了弹性mapreducer向导,加载了输入、输出、引导程序等。引导程序成功,但在执行时仍然出现错误。

这是我在步骤 1 的 stderr 中遇到的错误...

+ /etc/init.d/hadoop-state-pusher-control stop
+ PID_FILE=/mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ LOG_FILE=/mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ SVC_FILE=/mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ case $1 in
+ stop
+ echo 0
/etc/init.d/hadoop-state-pusher-control: line 35: /mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher: No such file or directory
+ /etc/init.d/hadoop-state-pusher-control start
+ PID_FILE=/mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ LOG_FILE=/mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ SVC_FILE=/mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ case $1 in
+ start
++ dirname /mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ sudo -u hadoop mkdir -p /mnt/var/lib/hadoop-state-pusher
+ echo 1
++ dirname /mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ sudo -u hadoop mkdir -p /mnt/var/run/hadoop-state-pusher
++ dirname /mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ sudo -u hadoop mkdir -p /mnt/var/log/hadoop-state-pusher
+ disown %1
+ sleep 5
+ sudo -u hadoop /usr/bin/hadoop-state-pusher -server --pidfile /mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ exit 0
Command exiting with ret '0'

这很神秘。这到底是什么意思?

好像安装什么东西有问题?其他哪个日志文件可能会提供一些信息,我应该在哪里查找?

我尝试了我找到的解决方案here https://groups.google.com/forum/#!msg/common-crawl/BAgYD8k4p7k/zFzCRWlT_7wJ,只是使实例更大,但这不起作用,同样的错误消息。


我正在查找错误的日志文件。有一个不同的(大约有 6 个?)实际上给了我一些有用的 python 调试信息。原来我用了一个string interpolation.format("of this kind {}, not this kind with a digit {1}".vars(a,b))这在 python

本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)

AWS Elastic mapreduce 似乎没有正确地将流媒体转换为 jar 的相关文章

随机推荐