搜索到
34
篇与
系统
的结果
-
Hadoop配置Hive Hadoop配置Hive{cloud title="" type="default" url="http://pan.000081.xyz/%E5%8D%9A%E5%AE%A2/%E5%A4%A7%E6%95%B0%E6%8D%AE/Hive" password=""/}0.环境前置Hadoop分布式部署完毕准备文件mysql-5.7.26-1.el7.x86_64.rpm-bundle.tarapache-hive-2.3.4-bin.tar.gzmysql-connector-java-5.1.46.jar1.安装Mysql1.检查是否存在与mysql冲突的包rpm -qa | grep mariadb #输出 mariadb-libs-5.5.56-2.el7.x86_642.卸载冲突包rpm -ev --nodeps mariadb-libs-5.5.56-2.el7.x86_643.解压Mysql安装包mkdir /mysql tar -xf mysql-5.7.26-1.el7.x86_64.rpm-bundle.tar -C /mysql/ cd /mysql/4.rpm按顺序安装Mysqlrpm -ivh mysql-community-common-5.7.26-1.el7.x86_64.rpm rpm -ivh mysql-community-libs-5.7.26-1.el7.x86_64.rpm rpm -ivh mysql-community-client-5.7.26-1.el7.x86_64.rpm rpm -ivh mysql-community-server-5.7.26-1.el7.x86_64.rpm5.启动mysqlservice mysqld start6.查看初始密码grep "password" /var/log/mysqld.log #输出 2022-03-24T12:04:39.603157Z 1 [Note] A temporary password is generated for root@localhost: 2yEFsa!sd2S77.进入Mysql并且修改密码mysql -uroot -p2yEFsa!sd2S7 #修改密码，因为我修改的是简单密码，所以修改相关参数 #修改validate_password_policy参数的值 mysql> set global validate_password_policy=0; Query OK, 0 rows affected (0.00 sec) #validate_password_length (密码长度) 参数默认为8，修改为1 mysql> set global validate_password_length=1; Query OK, 0 rows affected (0.00 sec) #修改密码 mysql> alter user 'root'@'localhost' identified by '000000'; Query OK, 0 rows affected (0.00 sec)8.开放远程登录mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '000000'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec)2.安装Hive1.解压Hivetar -xzf apache-hive-2.3.4-bin.tar.gz -C /2.添加环境变量vim /etc/profile #追加 export HIVE_HOME=/apache-hive-2.3.4-bin export PATH=$HIVE_HOME/bin:$PATH #刷新环境变量 source /etc/profile3.配置1.新建mysql数据库mysql -uroot -p000000 -e "create database hive_db";2.配置hive-site.xmlcd /apache-hive-2.3.4-bin/conf cp -r hive-default.xml.template hive-site.xml vim hive-site.xml #配置文件 <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://master:3306/hive_db?createDatabaseIfNotExist=true</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>000000</value> </property> <!--mysql 驱动--> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> ！如果改成mysql-connector-java8一定要把 com.mysql.jdbc.Driver改成com.mysql.cj.jdbc.Driver </property> <property> <name>hive.downloaded.resources.dir</name> <value>/apache-hive-2.3.4-bin/tmp</value> </property> <property> <name>hive.exec.local.scratchdir</name> <value>/apache-hive-2.3.4-bin/tmp/${hive.session.id}_resources</value> </property> <property> <name>hive.server2.enable.doAs</name> <value>false</value> </property> <property> <name>hive.metastore.schema.verification</name> <value>false</value> </property>3.将mysql驱动包放入hive的lib下pwd /apache-hive-2.3.4-bin/lib ls | grep mysql-connector-java-5.1.46.jar mysql-connector-java-5.1.46.jar4.启动Hadoopstart-all.sh5.配置hive-env.shcd /apache-hive-2.3.4-bin/conf cp -r hive-env.sh.template hive-env.sh vim hive-env.sh #追加 ---------------------------- export JAVA_HOME=/jdk1.8.0_191 export HADOOP_HOME=/hadoop-2.7.7 export HIVE_CONF_DIR=/apache-hive-2.3.4-bin/conf export HIVE_AUX_JARS_PATH=/apache-hive-2.3.4-bin/lib4.初始化并启动1.删除文件rm -rf /hadoop-2.7.7/share/hadoop/yarn/lib/jline-0.9.94.jar2.初始化Hiveschematool -initSchema -dbType mysql #出现以下即为成功 schemaTool completed -
Hadoop分布式搭建 Hadoop分布式搭建{cloud title="" type="default" url="http://pan.000081.xyz/%E5%8D%9A%E5%AE%A2/%E5%A4%A7%E6%95%B0%E6%8D%AE" password=""/}0.环境系统：centos7 64位处理器：8核内存：8G硬盘：200G数量：3台主机名IP地址master10.107.116.10slave110.107.116.11slave210.107.116.121.基础环境配置1.配置静态ipvi /etc/sysconfig/network-scripts/ifcfg-xxx BOOTPROTO=static ONBOOT="yes IPADDR=10.107.116.x NETMASK=255.255.255.0 GATEWAY=10.107.116.254 DNS1=114.114.114.1142.关闭防火墙和selinuxsystemctl stop firewalld systemctl disable firewalld setenforce 0 vi /etc/selinux/config #修改为 SELINUX=disabled3.修改主机名hostnamectl set-hostname 主机名 bash4.配置hosts文件vi /etc/hosts 加入 10.107.116.10 master 10.107.116.11 slave1 10.107.116.12 slave25.配置ssh免密登入#生成密钥 ssh-keygen #三台机互相执行配置免密 ssh-copy-id -i 主机6.配置时间同步yum install -y chrony vi /etc/chrony.conf master: ------------------------------------ #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst server master iburst allow 0.0.0.0/24 local stratum 10 ------------------------------------ slave(1和2) ------------------------------------ #server 0.centos.pool.ntp.org iburst #server 1.centos.pool.ntp.org iburst #server 2.centos.pool.ntp.org iburst #server 3.centos.pool.ntp.org iburst server master iburst ------------------------------------ systemctl restart chronyd && systemctl enable chronyd #查看时间同步源 chronyc sources -v #查看时间同步源状态 chronyc sourcestats -v #校准时间同步服务 chronyc tracking8.安装JDK环境#准备文件 jdk-8u191-linux-x64.tar.gz tar -xzf jdk-8u191-linux-x64.tar.gz -C / #添加环境变量 vi /etc/profile #追加 export JAVA_HOME=/jdk1.8.0_191 export PATH=$PATH:$JAVA_HOME/bin #刷新环境变量 source /etc/profile #验证安装 java -version #成功安装 java version "1.8.0_191" Java(TM) SE Runtime Environment (build 1.8.0_191-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)2.Hadoop安装所有节点1.解压Hadoop（hadoop-2.7.7.tar.gz）tar -xzf hadoop-2.7.7.tar.gz -C /2.添加环境变量vi /etc/profile #末尾追加 export HADOOP_HOME=/hadoop-2.7.7 export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin #刷新环境变量 source /etc/profile3.验证Hadoop环境变量hadoop version Hadoop 2.7.7 Subversion Unknown -r c1aad84bd27cd79c3d1a7dd58202a8c3ee1ed3ac Compiled by stevel on 2018-07-18T22:47Z Compiled with protoc 2.5.0 From source with checksum 792e15d20b12c74bd6f19a1fb886490 This command was run using /hadoop-2.7.7/share/hadoop/common/hadoop-common-2.7.7.jar3.Hadoop配置文件0.文件路径cd /hadoop-2.7.7/etc/hadoop/1.配置core-site.xmlvim core-site.xml ------------------------------ <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://master:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/root/hadoop/tmp</value> </property> </configuration>2.配置hdfs-site.xmlvim hdfs-site.xml ------------------------------ <configuration> <property> <name>dfs.replication</name> <value>3</value> </property> </configuration>3.配置mapred-site.xmlcp -r mapred-site.xml.template mapred-site.xml vim mapred-site.xml ------------------------------ <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration>4.配置yarn-site.xmlvim yarn-site.xml ------------------------------- <configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> </configuration>5.配置slavesvim slaves #清空改为 -------------------------------- master slave1 slave24.初始化Hadoopmasterhadoop namenode -format## 5.启动Hadoopmasterstart-all.sh如果JAVA_HOME报错，则所有节点hadoop-env.sh文件改为绝对路径，再次启动报错内容0.0.0.0: Error: JAVA_HOME is not set and could not be found.修改路径为绝对路径vim /hadoop-2.7.7/etc/hadoop/hadoop-env.sh export JAVA_HOME=${JAVA_HOME} 改为 export JAVA_HOME=/jdk1.8.0_1916.验证是否启动成功jps #出现 11858 NameNode 11495 ResourceManager 12167 SecondaryNameNode 12535 Jps 11993 DataNode 12411 NodeManager
-
-
centos7执行clear和top命令提示'xterm': unknown terminal type. centos7执行clear和top命令提示'xterm': unknown terminal type.[root@server ~]# clear 'xterm': unknown terminal type.解决办法临时办法(重新启动后失效)export TERM=vt100 export TEMCAP=$INFORMIXDIR/etc/termcap永久办法(去其他机子上scp -r /usr/share/terminfo文件覆盖)#另一台机 [root@centos ~]# cd /usr/share/ [root@centos share]# scp -r ./terminfo x.x.x.x:/usr/share/ root@x.x.x.x's password: #输入密码即可
-
Centos7通过arclight核心开我的世界1.17服务器 Centos7通过arclight核心开我的世界1.17服务器准备系统：Centos7系统 开服核心：arclight-forge-1.17.1-1.0.2.jar 运行环境：jdk-17_linux-x64_bin.tar.gz安装jdk17环境#解压jdk17软件包 [root@VM-16-16-centos mc]# tar -xzf jdk-17_linux-x64_bin.tar.gz #进入解压后文件夹，查看当前路径 [root@VM-16-16-centos mc]# cd jdk-17.0.2/ [root@VM-16-16-centos jdk-17.0.2]# pwd /root/mc/jdk-17.0.2 #添加环境变量 [root@VM-16-16-centos mc]# vim /etc/profile ####在最后一行追加环境变量#### export PATH=$PATH:/root/mc/jdk-17.0.2/bin ############################ #重新加载环境变量 [root@VM-16-16-centos jdk-17.0.2]# source /etc/profile #查看jdk17是否安装成功 [root@VM-16-16-centos jdk-17.0.2]# java -version java version "17.0.2" 2022-01-18 LTS Java(TM) SE Runtime Environment (build 17.0.2+8-LTS-86) Java HotSpot(TM) 64-Bit Server VM (build 17.0.2+8-LTS-86, mixed mode, sharing)运行开服核心[root@VM-16-16-centos mc]# cd /root/mc/ [root@VM-16-16-centos mc]# java -jar arclight-forge-1.17.1-1.0.2.jar等待开服核心所需文件下载完成The server installed successfully You can delete this installer file now if you wish执行启动脚本，同意声明并且再次启动[root@VM-16-16-centos mc]# ./run.sh #弹出未同意许可提示 [16:01:30] [main/WARN]: Failed to load eula.txt [16:01:30] [main/INFO]: You need to agree to the EULA in order to run the server. Go to eula.txt for more info. #修改eula.txt文件，改为同意 [root@VM-16-16-centos mc]# vi eula.txt #####修改前##### eula=false ################ #将false改为true #####修改后##### eula=true ################ #再次启动运行脚本 [root@VM-16-16-centos mc]# ./run.sh等待地图加载完成，开服成功[16:06:03] [Worker-Main-5/INFO]: Preparing spawn area: 90% [16:06:04] [Server thread/INFO]: Time elapsed: 59443 ms [16:06:04] [Server thread/INFO]: Done (69.396s)! For help, type "help"保持服务器在后台运行#输入stop关闭服务器返回bash stop [16:12:19] [Server thread/INFO]: Stopping the server [16:12:19] [Server thread/INFO]: Stopping server #yum安装screen软件 [root@VM-16-16-centos mc]# yum install -y screen #创建一个mc的窗口 [root@VM-16-16-centos mc]# screen -S mc #启动服务器 [root@VM-16-16-centos mc]# ./run.sh #直接断开终端即可，服务器会在后台运行 #重新进入服务器终端 #查看正在运行的面板 [root@VM-16-16-centos ~]# screen -ls There is a screen on: 10106.mc (Detached) 1 Socket in /var/run/screen/S-root. #进入运行的面板 [root@VM-16-16-centos mc]# screen -r mc常见问题检查端口是否被占用防火墙端口是否开放服务器安全策略是否开放{cloud title="" type="default" url="http://pan.000081.xyz/%E5%8D%9A%E5%AE%A2/linux/centos7/Centos7%E9%80%9A%E8%BF%87arclight%E6%A0%B8%E5%BF%83%E5%BC%80%E6%88%91%E7%9A%84%E4%B8%96%E7%95%8C1.17%E6%9C%8D%E5%8A%A1%E5%99%A8" password=""/}
