hadoop重新编译---安装系列二

2023-06-29 11:19:45 浏览数 (2)

yum -y install lzo-devel zlib-devel gcc autoconf automake libtool gcc-c openssl-devel ncurses-devel

ant maven protocolbuf findbugs cmake

1、Protobuf 编译安装

tar -zxvf protobuf-2.5.0.tar.gz

cd protobuf-2.5.0

./configure --prefix=/usr/local/protobuf

make

make install

2、Ant 安装

tar -zxvf apache-ant-1.9.2-bin.tar.gz

mv apache-ant-1.9.2/ /usr/local/ant

3、编译前的准备maven,官方下载地址

wget http://mirror.bit.edu.cn/apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.zip

maven 安装

tar -zxvf apache-maven-3.0.5-bin.tar.gz

mv apache-maven-3.0.5/ /usr/local/maven

4、findbugs 安装

tar -zxfv findbugs-2.0.2.tar.gz

mv findbugs-2.0.2/ /usr/local/findbugs

5、cmake 编译安装

tar -zvxf cmake-2.8.6.tar.gz

cd cmake-2.8.6

./bootstrap

gmake

make

make install

6、下载对应的64位的JDK

#java 安装 /usr/local/jdk1.7.0_45

7、配置环境

vi /etc/profile

#java

export JAVA_HOME=/usr/local/jdk1.7.0_45

export JRE_HOME=/usr/local/jdk1.7.0_45/jre

export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib:$JRE_HOME/lib

export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin

#maven

export MAVEN_HOME=/usr/local/maven

export MAVEN_OPTS="-Xms256m -Xmx512m"

export CLASSPATH=.:$CLASSPATH:$MAVEN_HOME/lib

export PATH=$PATH:$MAVEN_HOME/bin

#protobuf

export PROTOBUF_HOME=/usr/local/protobuf

export CLASSPATH=.:$CLASSPATH:$PROTOBUF_HOME/lib

export PATH=$PATH:$PROTOBUF_HOME/bin

#ant

export ANT_HOME=/usr/local/ant

export CLASSPATH=.:$CLASSPATH:$ANT_HOME/lib

export PATH=$PATH:$ANT_HOME/bin

#findbugs

export FINDBUGS_HOME=/usr/local/findbugs

export CLASSPATH=.:$CLASSPATH:$FINDBUGS_HOME/lib

export PATH=$PATH:$FINDBUGS_HOME/bin

source /etc/profile

8、如果是你32bit的机器,可以直接下载官方已经编译好的包,64bit的机子跑编译好的包跑不了。

由于maven国外服务器可能连不上,先给maven配置一下国内镜像,在maven目录下,conf/settings.xml,在<mirrors></mirros>里添加,原本的不要动

<mirror>

<id>nexus-osc</id>

<mirrorOf>*</mirrorOf>

<name>Nexusosc</name>

<url>http://maven.oschina.net/content/groups/public/</url>

</mirror>

同样,在<profiles></profiles>内新添加

<profile>

<id>jdk-1.7</id>

<activation>

<jdk>1.7</jdk>

</activation>

<repositories>

<repository>

<id>nexus</id>

<name>local private nexus</name>

<url>http://maven.oschina.net/content/groups/public/</url>

<releases>

<enabled>true</enabled>

</releases>

<snapshots>

<enabled>false</enabled>

</snapshots>

</repository>

</repositories>

<pluginRepositories>

<pluginRepository>

<id>nexus</id>

<name>local private nexus</name>

<url>http://maven.oschina.net/content/groups/public/</url>

<releases>

<enabled>true</enabled>

</releases>

<snapshots>

<enabled>false</enabled>

</snapshots>

</pluginRepository>

</pluginRepositories>

</profile>

hadoop源码地址下:

cd hadoop-maven-plugins

mvn install

9、首先官方下载hadoop源码

wget http://mirrors.cnnic.cn/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz

编译clean

cd hadoop2.2.0-src

mvn clean install –DskipTests

目前的2.2.0 的Source Code 压缩包解压出来的code有个bug 需要patch后才能编译。否则编译hadoop-auth 会提示错误:

vi /hadoop-2.2.0/hadoop-common-project/hadoop-auth/pom.xml

<dependency>

<groupid>org.mortbay.jetty</groupid>

<artifactid>jetty</artifactid>

<scope>test</scope>

</dependency>

在上面代码后添加下面代码

<dependency>

<groupid>org.mortbay.jetty</groupid>

<artifactid>jetty-util</artifactid>

<scope>test</scope>

</dependency>

注:不更改可能报下面错位

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure: ----------------------------------------------------------------------------------------------

ok,现在可以进行编译了

mvn clean package -Pdist,native -DskipTests -Dtar

如果出现错误可以加上-X显示错误详细信息:

mvn -X package -DskipTests -Pdist,native –Dtar

通过mvn package总是失败,可以考虑通过mvn compile来,如下:

mvn compile -Pdist,native -Dskiptests -Dtar

慢慢等吧!

0 人点赞