博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
hadoop完全分布式安装部署-笔记
阅读量:6262 次
发布时间:2019-06-22

本文共 4536 字,大约阅读时间需要 15 分钟。

规划:

[hadoop@db01 ~]$ cat /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.100.231         db01        #(namenode,datanode,JobHistoryServer,nodemanager)

192.168.100.232         db02        #(resourcemanager,datanode,nodemanager)
192.168.100.233         db03        #(SecondaryNameNode,datanode,nodemanager)
192.168.100.234         db04        #(nodemanager,datanode)
192.168.100.235         db05        #(nodemanager,datanode)

配置文件:

[hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/core-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

   

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

    <property>
            <name>fs.defaultFS</name>
            <value>hdfs://db01:9000</value>
    </property>

        <property>

                <name>hadoop.tmp.dir</name>
                <value>/usr/local/hadoop-2.5.0/data/tmp</value>
        </property>

        <property>

                <name>fs.trash.interval</name>
                <value>7000</value>
        </property>
</configuration>

[hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/hdfs-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

   

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

        <property>

                <name>dfs.namenode.secondary.http-address</name>
                <value>db03:50090</value>
        </property>

</configuration>

[hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/yarn-site.xml

<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

   

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<configuration>

    <property>

            <name>yarn.nodemanager.aux-services</name>
                <value>mapreduce_shuffle</value>
    </property>

        <property>

                <name>yarn.resourcemanager.hostname</name>
                <value>db02</value>
        </property>

    <property>

                <name>yarn.log-aggregation-enable</name>
                <value>true</value>
        </property>

        <property>

                <name>yarn.log-aggregation.retain-seconds</name>
                <value>600000</value>
        </property>
</configuration>

[hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

   

  Unless required by applicable law or agreed to in writing, software

  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

    <property>

            <name>mapreduce.framework.name</name>
                <value>yarn</value>
    </property>

        <property>

                <name>mapreduce.jobhistory.address</name>
                <value>db01:10020</value>
        </property>

        <property>

                <name>mapreduce.jobhistory.webapp.address</name>
                <value>db01:19888</value>
        </property>

</configuration>

[hadoop@db01 hadoop-2.5.0]$ cat etc/hadoop/slaves

db01
db02
db03
db04
db05

并且在(etc/hadoop/hadoop-env.sh  etc/hadoop/mapred-env.sh  etc/hadoop/yarn-env.sh)文件中配置相应的JAVA_HOME信息。

转载地址:http://dszpa.baihongyu.com/

你可能感兴趣的文章
一个简单的RMAN自动备份脚本
查看>>
转: 关于流量控制与令牌桶介绍
查看>>
Windows系统小知识
查看>>
变量使用self.foo还是_foo
查看>>
Codeforces Testing Round #12 B. Restaurant 贪心
查看>>
2015第47周五
查看>>
CSS-设置Footer始终在页面底部
查看>>
判断一个字符串同时出现几个字符的C#版本和JS版本
查看>>
asp.net获取客户端浏览器及主机信息
查看>>
jstack和线程dump分析
查看>>
如何使用Win8系统自带杀毒软件
查看>>
(转)No architectures to compile for (ONLY_ACTIVE_ARCH=YES, active arch=arm64, VA 解决办法
查看>>
GCD
查看>>
淘宝UWP--自定义图片缓存
查看>>
Javascript构造函数
查看>>
php--某个字符在字符串中的位置比较
查看>>
一个类有两个方法,其中一个是同步的,另一个是非同步的; 现在又两个线程A和B,请问:当线程A访问此类的同步方法时,线程B是否能访问此类的非同步方法?...
查看>>
[LeetCode] Maximum Product of Word Lengths 单词长度的最大积
查看>>
socket通信中select函数的使用和解释
查看>>
JAVA Map集合类简介
查看>>