hadoop单机版部署

1.下载hadoop

wget --no-check-certificate  https://mirrors.bfsu.edu.cn/apache/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz

2.解压重命名

tar -zxvf hadoop-3.3.1.tar.gz

mv hadoop-3.3.1.tar.gz  hadoop

3.编辑hosts

vim /etc/hosts

172.17.1.1 hadoop925

4.进入配置 cd /data/hadoop/etc/hadoop/

 5.vim hadoop-env.sh 添加JAVA

export JAVA_HOME=/usr/java/jdk1.8.0_121

 

6.vim core-site.xml

<configuration>
 <property>
        <name>hadoop.tmp.dir</name>
        <value>/datah/data/hadoop/work/hadoop_tmp</value>
        <description>A base for other temporary directories.</description>
    </property>
   <property>
        <name>fs.default.name</name>
        <value>hdfs://hadoop925:8020</value>
    </property>
</configuration>

7.vim hdfs-site.xml

<configuration>

    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>/data/hadoop/hdpdata/name/</value>
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>/data/hadoop/hdpdata/data/</value>
    </property>
    <property>
        <name>dfs.permissions</name>
        <value>false</value>
    </property>
    <property>
        <name>dfs.namenode.http-address</name>
        <value>hadoop925:9870</value>
    </property>

</configuration>
 

8.vim yarn-site.xml

<configuration>

 <property>
      <name>yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage</name>
      <value>98.0</value>
  </property>


</configuration>

9.进入/data/hadoop/sbin ，相关sh 文件配置添加到顶部

9.1  vim start-dfs.sh 和 vim stop-dfs.sh 顶部添加配置

#!/usr/bin/env bash

HDFS_NAMENODE_USER=root

HDFS_DATANODE_USER=root

HDFS_SECONDARYNAMENODE_USER=root

YARN_RESOURCEMANAGER_USER=root

YARN_NODEMANAGER_USER=root

9.2  vim start-yarn.sh 和 vim stop-yarn.sh顶部添加配置

#!/usr/bin/env bash

YARN_RESOURCEMANAGER_USER=root

HADOOP_SECURE_DN_USER=yarn

YARN_NODEMANAGER_USER=root
 

10.cd bin 目录编译配置

./hadoop namenode -format

11. 启动/停用hadoop

./start-all.sh

./start-all.sh