Hadoop分布式搭建 Hadoop分布式搭建{cloud title="" type="default" url="http://pan.000081.xyz/%E5%8D%9A%E5%AE%A2/%E5%A4%A7%E6%95%B0%E6%8D%AE" password=""/}0.环境系统：centos7 64位处理器：8核内存：8G硬盘：200G数量：3台主机名IP地址master10.107.116.10slave110.107.116.11slave210.107.116.121.基础环境配置1.配置静态ipvi /etc/sysconfig/network-scripts/ifcfg-xxx BOOTPROTO=static ONBOOT="yes IPADDR=10.107.116.x NETMASK=255.255.255.0 GATEWAY=10.107.116.254 DNS1=114.114.114.1142.关闭防火墙和selinuxsystemctl stop firewalld systemctl disable firewalld setenforce 0 vi /etc/selinux/config #修改为 SELINUX=disabled3.修改主机名hostnamectl set-hostname 主机名 bash4.配置hosts文件vi /etc/hosts 加入 10.107.116.10 master 10.107.116.11 slave1 10.107.116.12 slave25.配置ssh免密登入#生成密钥 ssh-keygen #三台机互相执行配置免密 ssh-copy-id -i 主机6.配置时间同步yum install -y chrony vi /etc/chrony.conf master: ------------------------------------ #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst server master iburst allow 0.0.0.0/24 local stratum 10 ------------------------------------ slave(1和2) ------------------------------------ #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst server master iburst ------------------------------------ systemctl restart chronyd && systemctl enable chronyd #查看时间同步源 chronyc sources -v #查看时间同步源状态 chronyc sourcestats -v #校准时间同步服务 chronyc tracking8.安装JDK环境#准备文件 jdk-8u191-linux-x64.tar.gz tar -xzf jdk-8u191-linux-x64.tar.gz -C / #添加环境变量 vi /etc/profile #追加 export JAVA_HOME=/jdk1.8.0_191 export PATH=$PATH:$JAVA_HOME/bin #刷新环境变量 source /etc/profile #验证安装 java -version #成功安装 java version "1.8.0_191" Java(TM) SE Runtime Environment (build 1.8.0_191-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)2.Hadoop安装所有节点1.解压Hadoop（hadoop-2.7.7.tar.gz）tar -xzf hadoop-2.7.7.tar.gz -C /2.添加环境变量vi /etc/profile #末尾追加 export HADOOP_HOME=/hadoop-2.7.7 export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin #刷新环境变量 source /etc/profile3.验证Hadoop环境变量hadoop version Hadoop 2.7.7 Subversion Unknown -r c1aad84bd27cd79c3d1a7dd58202a8c3ee1ed3ac Compiled by stevel on 2018-07-18T22:47Z Compiled with protoc 2.5.0 From source with checksum 792e15d20b12c74bd6f19a1fb886490 This command was run using /hadoop-2.7.7/share/hadoop/common/hadoop-common-2.7.7.jar3.Hadoop配置文件0.文件路径cd /hadoop-2.7.7/etc/hadoop/1.配置core-site.xmlvim core-site.xml ------------------------------ <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://master:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/root/hadoop/tmp</value> </property> </configuration>2.配置hdfs-site.xmlvim hdfs-site.xml ------------------------------ <configuration> <property> <name>dfs.replication</name> <value>3</value> </property> </configuration>3.配置mapred-site.xmlcp -r mapred-site.xml.template mapred-site.xml vim mapred-site.xml ------------------------------ <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration>4.配置yarn-site.xmlvim yarn-site.xml ------------------------------- <configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> </configuration>5.配置slavesvim slaves #清空改为 -------------------------------- master slave1 slave24.初始化Hadoopmasterhadoop namenode -format## 5.启动Hadoopmasterstart-all.sh如果JAVA_HOME报错，则所有节点hadoop-env.sh文件改为绝对路径，再次启动报错内容0.0.0.0: Error: JAVA_HOME is not set and could not be found.修改路径为绝对路径vim /hadoop-2.7.7/etc/hadoop/hadoop-env.sh export JAVA_HOME=${JAVA_HOME} 改为 export JAVA_HOME=/jdk1.8.0_1916.验证是否启动成功jps #出现 11858 NameNode 11495 ResourceManager 12167 SecondaryNameNode 12535 Jps 11993 DataNode 12411 NodeManager