hadoop学习笔记之二：1.0.1搭建集群

       安装环境

   

写道

两台主机：
192.168.1.2 bigdata2
192.168.1.4 bigdata3

 

   1  安装JDK （两台主机安装一个版本的JDK）

  

[root@bigdata2 bin]# echo $JAVA_HOME
/opt/java/jdk1.6.0_45

  

 

   2 安装SSH，配置面密码登录（bigdata2 可以免密码登录到bigdata3）

   在bigdata2上面执行如下指令

 

ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

   id_rsa.pub复制到bigdata3机器下的.ssh/authorized_keys文件里

 

 

scp  id_rsa.pub root@bigdata3:/root/.ssh/authorized_keys
ssh bigdata3

 

 

   3 进入conf目录，修改配置文件

    core-site.xml

  

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
        <property>
                <name>fs.default.name</name>
                <value>hdfs://bigdata2:9000</value>
        </property>

         <property>
                <name>hadoop.tmp.dir</name>
                <value>/root/u/tmp/hadoop-${user.name}</value>
        </property>
</configuration>

  假如/root/u/tmp目录不存在，需要建立起来 (两台机器上目录位置相同)

 

  

mkdir -p /root/u/tmp

    hdfs-site.xml

 

  

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
       <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
</configuration>

 mapred-site.xml

 

 

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
     <property>
                <name>mapred.job.tracker</name>
                <value>bigdata2:9001</value>
        </property>
</configuration>

   masters

[root@bigdata2 conf]# more masters 
bigdata2

   slaves

   

[root@bigdata2 conf]# more slaves 
bigdata3

   修改hadoop-env.sh 设置JAVA_HOME

   

# The java implementation to use.  Required.
# export JAVA_HOME=/usr/lib/j2sdk1.5-sun
  export JAVA_HOME=/opt/java/jdk1.6.0_45

# Extra Java CLASSPATH elements.  Optional.
# export HADOOP_CLASSPATH=

# The maximum amount of heap to use, in MB. Default is 1000.
# export HADOOP_HEAPSIZE=2000

# Extra Java runtime options.  Empty by default.
# export HADOOP_OPTS=-server

 

   4  拷贝配置好的hadoop目录到bigdata3，放到与bigdata2相同的目录

 

   5 格式化HDFS文件系统

 

  /bin/hadoop namenode -format

   6 启动

  

./bin/start-all.sh

 

  7 jps 检测

  

[root@bigdata2 conf]# jps
6096 Jps
5452 NameNode
5758 JobTracker
5630 SecondaryNameNode

 8 进入bigdata3

  

[root@bigdata2 conf]# ssh bigdata3
Last login: Thu Nov 27 11:36:27 2014 from bigdata2
[root@bigdata3 ~]# jps
19584 DataNode
20226 Jps
19713 TaskTracker

 

 9 访问50070和50030端口查看集群和HDFS的状态