问题:
每个人安装hadoop时,都可能会遇见这个问题“WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable“,然后转手就去百度。也不知道网上怎么传的,都说是版本问题,各种编译就上来了,我就日了,这坑了多少人,几天的时间就这么被坑了。
正解:
作为程序员,遇见问题我们第一时间应该是去看日志的,伸手党们习惯直接百度,然后一步一步走弯路。
由于hadoop默认的日志级别较低,执行命令export HADOOP_ROOT_LOGGER=DEBUG,console改一下,会发现日志里面写的很清晰:
代码语言:javascript复制17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of successful kerberos logins and latency (milliseconds)])
17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[Rate of failed kerberos logins and latency (milliseconds)])
17/11/06 18:53:16 DEBUG lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.getGroups with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, valueName=Time, value=[GetGroups])
17/11/06 18:53:16 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group related metrics
17/11/06 18:53:16 DEBUG security.Groups: Creating new Groups object
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /hadoop/hadoop-2.7.1/lib/native/libhadoop.so.1.0.0: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /hadoop/hadoop-2.7.1/lib/native/libhadoop.so.1.0.0)
17/11/06 18:53:16 DEBUG util.NativeCodeLoader: java.library.path=/hadoop/hadoop-2.7.1/lib/native
17/11/06 18:53:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/11/06 18:53:16 DEBUG util.PerformanceAdvisory: Falling back to shell based
17/11/06 18:53:16 DEBUG security.JniBasedUnixGroupsMappingWithFallback: Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
17/11/06 18:53:16 DEBUG security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000; warningDeltaMs=5000
17/11/06 18:53:17 DEBUG security.UserGroupInformation: hadoop login
17/11/06 18:53:17 DEBUG security.UserGroupInformation: hadoop login commit
17/11/06 18:53:17 DEBUG security.UserGroupInformation: using local user:UnixPrincipal: root
17/11/06 18:53:17 DEBUG security.UserGroupInformation: Using user: "UnixPrincipal: root" with name root
17/11/06 18:53:17 DEBUG security.UserGroupInformation: User entry: "root"
17/11/06 18:53:17 DEBUG security.UserGroupInformation: UGI loginUser:root (auth:SIMPLE)
17/11/06 18:53:17 DEBUG security.UserGroupInformation: PrivilegedAction as:root (auth:SIMPLE) from:org.apache.hadoop.hdfs.tools.GetConf.run(GetConf.java:315)
一眼就能看出问题的所在,glibc问题,人家需要的版本高,你应该升级一下。
使用命令:strings /lib64/libc.so.6 |grep GLIBC_
代码语言:javascript复制[root@hadoop02 sbin]# strings /lib64/libc.so.6 |grep GLIBC_
GLIBC_2.2.5
GLIBC_2.2.6
GLIBC_2.3
GLIBC_2.3.2
GLIBC_2.3.3
GLIBC_2.3.4
GLIBC_2.4
GLIBC_2.5
GLIBC_2.6
GLIBC_2.7
GLIBC_2.8
GLIBC_2.9
GLIBC_2.10
GLIBC_2.11
GLIBC_2.12
GLIBC_PRIVATE
发现自己的版本低,解决办法就是升级,引用别人的劳动成果。
上一篇遇见问题,参考下面解决了。
http://www.linuxdiyf.com/linux/23702.html