s_第311页_有道词典
& 2011 网易公司 京ICP证080268号9276人阅读
版本多节点集群安装及测试
作者：雨水， 日期：， CSDN博客：
摘要：本文记录了Hadoop2.2.0版本多节点集群安装过程，并做了基本配置，启动运行和测试了一个单词统计例子。
环境说明：
基于Windows下的中的ubuntu12.04-64server安装，先把基础软件安装到一个虚拟机中，然后拷贝两份再配置下即可。三台机器的分工如下：
Hadoop1(Master):&NameNode/ResouceManager
Hadoop2(Slave):DataNode/NodeManager
&Hadoop3(Slave): DataNode/NodeManager
假定三台虚拟机的IP地址如下，后面会用到。
Hadoop1:192.168.128.130
Hadoop2:192.168.128.131
Hadoop3:192.168.128.132
一：环境准备：
下载免费的VMware Player并安装好; 下载 免费的Ubuntu 12.04 server版并在VMware中安装好；
二：基础安装：
执行如下命令升级部分软件和把ssh安装好
&& (1)sudo apt-
&& (2)sudo apt-
&& (3)sudo apt-get install openssh-
安装Oracle JDK
通过webupd8team自动安装，执行命令如下：
(1)&&&sudo apt-get install python-software-properties
(2)&&&sudo add-apt-repository ppa:webupd8team/java
(3)&&&sudo apt-get update
(4)&&&sudo apt-get install oracle-java6-installer
创建hadoop用户
(1)&&&sudo addgroup hadoop
(2)&&&sudo adduser --ingroup hadoop hduser
编辑/etc/sudoers编辑文件，在root ALL=(ALL)ALL行下添加hduser ALL=(ALL)ALL。如果不添加这行，hduser将不能执行sudo操作。
注：以下操作均用hduser用户登录后操作。
三：公共安装：
注：以下操作以hduser登录进行操作。
下载Hadoop2.2.0版本
(1)&&&cd /home/hduser
(2)&&&$&wget/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.tar.gz
(3)&&&$ tar zxf hadoop-2.2.0.tar.gz
(4)&&&$&mv hadoop-2.2.0 hadoop
配置Hadoop
(1)&&&配置/home/hduser/hadoop/etc/hadoop/hadoop-env.sh
替换exportJAVA_HOME=${JAVA_HOME}为如下：
export JAVA_HOME=/usr/lib/jvm/java-6-oracle
(2) 配置/home/hduser/hadoop/etc/hadoop/core-site.xml，
在&configuration&中添加如下：
&property&
& &name&hadoop.tmp.dir&/name&
&&value&/home/hduser/hadoop/tmp/hadoop-${user.name}&/value&
&&description&A base for other temporary directories.&/description&
&/property&
&property&
&&name&fs.default.name&/name&
&&value&hdfs://192.168.128.130:8010&/value&
&&description&The name of the default file system.& A URI whose
& schemeand authority determine the FileSystem implementation.& The
& uri'sscheme determines the config property (fs.SCHEME.impl) naming
& theFileSystem implementation class.& Theuri's authority is used to
&determine the host, port, etc. for a filesystem.&/description&
&/property&
注意：以下两点务必确保正确，否则后面会出错。
a. 需执行mkdird home/hduser/hadoop/tmp创建这个临时目录;
b. 这个fd.default.name值的IP地址为NameNode的地址，即Hadoop1。
配置/home/hduser/hadoop/etc/hadoop/mapred-site.xml
(1)&&&mv /home/hduser/hadoop/etc/hadoop/mapred-site.xml.template/home/hduser/hadoop/etc/hadoop/mapred-site.xml
(2)&&&在&configuration&中添加如下：
&property&
&&name&mapred.job.tracker&/name&
&&value&192.168.128.130:54311&/value&
&&description&The host and port that theMapReduce job tracker runs
&at.& If &local&, thenjobs are runin-process as a single map
&and reducetask.
&&/description&
&/property&
配置/home/hduser/hadoop/etc/hadoop/hdfs-site.xml
在&configuration&中添加如下：
&property&
&&name&dfs.replication&/name&
&&value&2&/value&
&&description&Default block replication.
& Theactual number of replications can be specified when the file is created.
& Thedefault is used if replication is not specified in create time.
& &/description&
&/property&
四：整体安装
将上面安装和配置好的虚拟机拷贝两份，即Hadoop2和Hadoop3。 分别修改三台虚拟机的/etc/hostname中的内容改为相应的主机名，即
hadoop1的hostname为hadoop1，其他类推。
修改完成后需要重启，并通过命令hostname确认已经生效。
分别检查并修改三台虚拟机的/etc/hosts中的内容，确保包含如下配置：
192.168.128.130 hadoop1
192.168.128.131 hadoop2
192.168.128.132 hadoop3
为三台虚拟机之间建立SSH信任以便是实现无需密码登陆。
(1)&&&将以下命令分别在三台机子上做一遍：
$echo&& & .ssh/authorized_keys
$ssh-keygen-t rsa -P &&
&$cat.ssh/id_rsa.pub &&.ssh/authorized_keys
(2)&&&分别将各台机子上的.ssh/id_rsa.pub的内容追加到其他两台的.ssh/authorized_keys中，这样三台机子相互访问就不需要输入密码了。可通过ssh hadoop1来测试。
分别修改各台机子的$HADOOP_HOME/etc/hadoop/slaves，这里$HADOOP_HOME为你的hadoop安装目录。Slaves的内容如下：
五：运行Hadoop
注：所有的运行只需要在hadoop1的master节点即可。系统会自动登录到其他两台去启动相应的节点。
在初次运行Hadoop的时候需要初始化Hadoop文件系统，命令如下：
$./hdfs namenode -format
如果执行成功，你会在日志中(倒数几行)找到如下成功的提示信息：
common.Storage: Storage directory /home/hduser/hadoop/tmp/hadoop-hduser/dfs/namehas been successfully formatted.
运行命令如下：
$cd home/hduser/hadoop/sbin/
$./start-dfs.sh
$./start-yarn.sh
启动完之后可分别用jps命令查看各个机子的进程是否正常，如下：
hduser@hadoop1:~/hadoop$jps
1777 ResourceManager
1464 NameNode
1618 SecondaryNameNode
hduser@hadoop2:~$ jps
1264 DataNode
1344 NodeManager
hduser@hadoop3:~$ jps
1289 NodeManager
1209 DataNode
还可以通过bin/hdfs dfsadmin -report查看dfs是否正常，相应如下：
hduser@hadoop1:~/hadoop$ bin/hdfs dfsadmin -report
Configured Capacity: (37.88 GB)
Present Capacity: (31.98 GB)
DFS Remaining: (31.98 GB)
DFS Used: .13 MB)
DFS Used%: 0.01%
Under replicatedblocks: 0
Blocks with corruptreplicas: 0
Missing blocks: 0
-------------------------------------------------
Datanodes available:2 (2 total, 0 dead)
Live datanodes:
Name:192.168.128.131:50010 (hadoop2)
Hostname: hadoop2
Decommission Status: Normal
Configured Capacity: (18.94 GB)
DFS Used: .06 MB)
Non DFS Used: (2.95 GB)
DFS Remaining: (15.99 GB)
DFS Used%: 0.01%
DFS Remaining%:84.43%
Last contact: MonOct 28 19:12:16 PDT 2013
Name:192.168.128.132:50010 (hadoop3)
Hostname: hadoop3
Decommission Status: Normal
Configured Capacity: (18.94 GB)
DFS Used: .06 MB)
Non DFS Used: (2.95 GB)
DFS Remaining: (15.99 GB)
DFS Used%: 0.01%
DFS Remaining%:84.43%
Last contact: MonOct 28 19:12:16 PDT 2013
六：查看Hadoop资源管理器
，将其中的192.168.128.130替换为你的Hadoop1的IP地址。
七：测试Hadoop
cd /home/hduser
$wget http://www.gutenberg.org/cache/epub/20417/pg20417.txt
$cd hadoop
$ bin/hdfs dfs -mkdir /tmp
$ bin/hdfs dfs -copyFromLocal /home/hduser/pg20417.txt /tmp
bin/hdfs dfs -ls /tmp
$bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jarwordcount /tmp/ /tmp-output
如果一切正常的话，会输入相应的结果，可以从屏幕输出看到。
八：停止Hadoop
若停止hadoop，依次运行如下命令：
$./stop-yarn.sh
$./stop-dfs.sh
九：集群安装与单机安装的区别分析
core-site.xml中配置的fs.default.name值的IP地址必须为Master节点，本文为Hadoop1节点； hdfs-site.xml中配置的dfs.replication值需要与实际的DataNode节点数一致，本文为2； mapred-site.xml中配置的mapred.job.tracker值的IP地址必须为Master节点，本文为Hadoop1节点； 文件必须配置上实际的slaves节点，本文为hadoop2和hadoop3；
每个主机的/etc/hostname和/etc/hostname必须做相应的配置以方便集群内部相互识别； 必须在集群内部用ssh建立起信任。
以上安装过程中还是出现了一些问题，但基本都通过baidu和google解决了。有个错误花费了较多时间，这里记录下，以供参考。
错误现象：13/10/ WARN hdfs.DFSClient: DataStreamer Exception org.apache.hadoop.ipc.RemoteException(java.io.IOException):File /tmp/pg20417.txt._COPYING_ could only be replicated to 0 nodes instead ofminReplication (=1).& There are 0datanode(s)
running and no node(s) are excluded in this operation.
发生地方：执行bin/hdfs dfs -copyFromLocal /home/hduser/pg20417.txt /tmp时
原因定位：后来经过反复查看，是因为fs.default.name的值中的IP地址配置成 localhost了，导致系统找不到hdfs.是在datanode的日志中发现这个错误的，日志如下：
07:33:55,963 WARNorg.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to server:localhost/127.0.0.1:8010
解决办法：将fs.default.name中的IP地址改为192.168.128.130，即你的master节点的IP地址。
十：参考资料
* 以上用户言论只代表其个人观点，不代表CSDN网站的观点或立场
访问：376119次
积分：5032
积分：5032
排名：第2118名
原创：136篇
评论：193条
(1)(1)(1)(1)(2)(5)(1)(4)(5)(3)(2)(1)(1)(2)(4)(1)(1)(1)(2)(2)(2)(1)(1)(2)(1)(3)(3)(2)(1)(1)(3)(1)(6)(1)(2)(2)(1)(1)(3)(3)(2)(3)(1)(3)(1)(4)(2)(5)(2)(1)(1)(3)(3)(3)(4)(6)(5)(12)(1)(1)(1)Continuous monitoring and control of plasma glucose during operation for removal of insulinomas.
- PubMed - NCBI
The NCBI web site requires JavaScript to function.
FormatSummarySummary (text)AbstractAbstract (text)MEDLINEXMLPMID ListChoose DestinationFileClipboardCollectionsE-mailOrderMy BibliographyCitation managerFormatSummary (text)Abstract (text)MEDLINEXMLPMID ListCSVCreate File1 selected item: 222000FormatSummarySummary (text)AbstractAbstract (text)MEDLINEXMLPMID ListMeSH and Other DataE-mailSubjectAdditional textE-mailAdd to ClipboardAdd to CollectionsOrder articlesAdd to My BibliographyGenerate a file for use with external citation management software.Create File
):702-7.Continuous monitoring and control of plasma glucose during operation for removal of insulinomas., , , , .AbstractA glucose-controlled insulin and glucose infusion system (Biostator system, Miles Laboratories Inc., Elkhart, Ind.) was used during operation for removal of four suspected insulinomas. In two patients the Biostator system continuously monitored plasma glucose levels and, through a computer-controlled feedback mechanism, automatically infused insulin or glucose as needed. In this way plasma glucose could be kept at approximately 110 mg/dl throughout the procedure. In one patient a rise in insulin infusion rate and a fall in glucose infusion rate followed removal of the tumor. In the other, absence of these findings was consistent with the clinical suspicion that the source of hypoglycemia had not been removed. In two other patients the Biostator system continuously monitored the patient's plasma glucose, without feedback-controlled administration of insulin or glucose. This demonstrated that, in these patients, tumor manipulation and removal did not lead to severe hypoglycemia, and thus eliminated any need to prophylactically infuse large amounts of glucose. Successful tumor removal was followed by a rise in blood glucose levels in both of these individuals. The glucose-controlled insulin and glucose infusion system appears to be a useful instrument for intraoperative management of patients with suspected insulinomas.PMID: 222000
[PubMed - indexed for MEDLINE]
Publication TypesMeSH TermsSubstancesOther Literature SourcesMedicalMolecular Biology Databases
Supplemental Content
External link. Please review our .您现在的位置： >
> TLV2785AIPWR (Texas Instruments, Inc.) FAMILY OF 1.8V HIGH SPEED RAIL TO RAIL INPUT OUTPUT OPERATIONAL AMPLIFIERS WITH SHUTDOWN PDF资料下载
Orderable DeviceStatus(1)PackageTypeSOICPackageDrawingDPins PackageQty75Eco Plan(2)Lead/Ball FinishMSL Peak Temp(3)TLV2780AIDACTIVE8Pb-Free(RoHS)Pb-Free(RoHS)Pb-Free(RoHS)CU NIPDAULevel-2-260C-1YEAR/Level-1-220C-UNLIMLevel-2-260C-1YEAR/Level-1-220C-UNLIMLevel-2-260C-1YEAR/Level-1-220C-UNLIMLevel-1-260C-UNLIMTLV2780AIDRACTIVESOICD82500CU NIPDAUTLV2780CDACTIVESOICD875CU NIPDAUTLV2780CDBVRACTIVESOT-23DBV63000 Green (RoHS &no Sb/Br)250Green (RoHS &no Sb/Br)250Green (RoHS &no Sb/Br)2500Pb-Free(RoHS)75Pb-Free(RoHS)3000 Green (RoHS &no Sb/Br)3000 Green (RoHS &no Sb/Br)250Green (RoHS &no Sb/Br)2500Pb-Free(RoHS)50Pb-Free(RoHS)75Pb-Free(RoHS)2500Pb-Free(RoHS)75Pb-Free(RoHS)3000 Green (RoHS &no Sb/Br)3000 Green (RoHS &no Sb/Br)250Green (RoHS &no Sb/Br)2500Pb-Free(RoHS)75Pb-Free(RoHS)3000 Green (RoHS &no Sb/Br)3000 Green (RoHS &no Sb/Br)250Green (RoHS &no Sb/Br)2500Pb-Free(RoHS)CU NIPDAUTLV2780CDBVTACTIVESOT-23DBV6CU NIPDAULevel-1-260C-UNLIMTLV2780CDBVTG4ACTIVESOT-23DBV6CU NIPDAULevel-1-260C-UNLIMTLV2780CDRACTIVESOICD8CU NIPDAULevel-2-260C-1YEAR/Level-1-220C-UNLIMLevel-2-260C-1YEAR/Level-1-220C-UNLIMLevel-1-260C-UNLIMTLV2780IDACTIVESOICD8CU NIPDAUTLV2780IDBVRACTIVESOT-23DBV6CU NIPDAUTLV2780IDBVRG4ACTIVESOT-23DBV6CU NIPDAULevel-1-260C-UNLIMTLV2780IDBVTACTIVESOT-23DBV6CU NIPDAULevel-1-260C-UNLIMTLV2780IDRACTIVESOICD8CU NIPDAULevel-2-260C-1YEAR/Level-1-220C-UNLIMLevel-NC-NC-NCTLV2780IPACTIVEPDIPP8CU NIPDAUTLV2781AIDACTIVESOICD8CU NIPDAULevel-2-260C-1YEAR/Level-1-220C-UNLIMLevel-2-260C-1YEAR/Level-1-220C-UNLIMLevel-2-260C-1YEAR/Level-1-220C-UNLIMLevel-1-260C-UNLIMTLV2781AIDRACTIVESOICD8CU NIPDAUTLV2781CDACTIVESOICD8CU NIPDAUTLV2781CDBVRACTIVESOT-23DBV5CU NIPDAUTLV2781CDBVRG4ACTIVESOT-23DBV5CU NIPDAULevel-1-260C-UNLIMTLV2781CDBVTACTIVESOT-23DBV5CU NIPDAULevel-1-260C-UNLIMTLV2781CDRACTIVESOICD8CU NIPDAULevel-2-260C-1YEAR/Level-1-220C-UNLIMLevel-2-260C-1YEAR/Level-1-220C-UNLIMLevel-1-260C-UNLIMTLV2781IDACTIVESOICD8CU NIPDAUTLV2781IDBVRACTIVESOT-23DBV5CU NIPDAUTLV2781IDBVRG4ACTIVESOT-23DBV5CU NIPDAULevel-1-260C-UNLIMTLV2781IDBVTACTIVESOT-23DBV5CU NIPDAULevel-1-260C-UNLIMTLV2781IDRACTIVESOICD8CU NIPDAULevel-2-260C-1YEAR/Level-1-220C-UNLIMPACKAGE OPTION ADDENDUM22-Feb-2005Addendum-Page 1
FAMILY OF 1.8 V HIGH-SPEED RAIL-TO-RAIL INPUT/OUTPUT OPERATIONAL AMPLIFIERS WITH SHUTDOWN
FAMILY OF 1.8V HIGH SPEED RAIL TO RAIL INPUT OUTPUT OPERATIONAL AMPLIFIERS WITH SHUTDOWN
FAMILY OF 1.8V HIGH SPEED RAIL TO RAIL INPUT OUTPUT OPERATIONAL AMPLIFIERS WITH SHUTDOWN
FAMILY OF 1.8 V HIGH-SPEED RAIL-TO-RAIL INPUT/OUTPUT OPERATIONAL AMPLIFIERS WITH SHUTDOWN
FAMILY OF 1.8 V HIGH-SPEED RAIL-TO-RAIL INPUT/OUTPUT OPERATIONAL AMPLIFIERS WITH SHUTDOWN
相关代理商/技术参数
制造商:Texas Instruments 功能描述:OP Amp Quad GP R-R I/O 制造商:Rochester Electronics LLC 功能描述:- Bulk
制造商:Texas Instruments 功能描述:OP Amp Quad GP R-R I/O 制造商:Rochester Electronics LLC 功能描述:- Bulk
功能描述:运算放大器 - 运放 Quad 1.8V RRIO 8MHz Amp w/Shtdwn RoHS:否 制造商:STMicroelectronics 通道数量:4 共模抑制比（最小值）:63 dB 输入补偿电压:1 mV 输入偏流（最大值）:10 pA 工作电源电压:2.7 V to 5.5 V 安装风格:SMD/SMT 封装 / 箱体:QFN-16 转换速度:0.89 V/us 关闭:No 输出电流:55 mA 最大工作温度:+ 125 C 封装:Reel
功能描述:运算放大器 - 运放 Quad 1.8V RRIO 8MHz Amp w/Shtdwn RoHS:否 制造商:STMicroelectronics 通道数量:4 共模抑制比（最小值）:63 dB 输入补偿电压:1 mV 输入偏流（最大值）:10 pA 工作电源电压:2.7 V to 5.5 V 安装风格:SMD/SMT 封装 / 箱体:QFN-16 转换速度:0.89 V/us 关闭:No 输出电流:55 mA 最大工作温度:+ 125 C 封装:Reel
功能描述:运算放大器 - 运放 Quad 1.8V RRIO 8MHz Amp w/Shtdwn RoHS:否 制造商:STMicroelectronics 通道数量:4 共模抑制比（最小值）:63 dB 输入补偿电压:1 mV 输入偏流（最大值）:10 pA 工作电源电压:2.7 V to 5.5 V 安装风格:SMD/SMT 封装 / 箱体:QFN-16 转换速度:0.89 V/us 关闭:No 输出电流:55 mA 最大工作温度:+ 125 C 封装:Reel
(若只采购一条型号，填写一行即可)
添加更多采购