Mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Sprague <sprag...@gmail.com>
Subject config issues
Date Sun, 18 Sep 2016 01:03:57 GMT
here's the info i have where i'm obviously missing something.

a) i've read and re-read the tez install instructions. still no dice.
b) we're running yarn.
c) we're also running pepperdata.
d) trying to get tez 0.8.4 working.
e) i think the client side is pretty easy. get the tez jars in the
classpath.

however on the server side i've hit a brick wall.



from the datanode log:

# i think this looks good.

2016-09-17 15:56:37,483 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource:
Resource *hdfs://dwrdevnn1.sv2.trulia.com:8020/apps/tez-0.8.4/apache-tez-0.8.4-bin.tar.gz
<http://dwrdevnn1.sv2.trulia.com:8020/apps/tez-0.8.4/apache-tez-0.8.4-bin.tar.gz>*(->/storage4/hadoop/yarn/local/filecache/10/apache-tez-0.8.4-bin.tar.gz)
transitioned from DOWNLOADING to LOCALIZED
2016-09-17 15:56:37,484 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Container container_1474152944798_0001_02_000001 transitioned from
LOCALIZING to LOCALIZED
2016-09-17 15:56:37,509 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
Container container_1474152944798_0001_02_000001 transitioned from
LOCALIZED to RUNNING

# here's the command/container that gets run

Command $JAVA_HOME/bin/java  -Xmx819m -Djava.io.tmpdir=$PWD/tmp -server
-Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN
-XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA
-XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/
-Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator
-Dlog4j.configuration=tez-container-log4j.properties
-Dyarn.app.container.log.dir=/storage5/hadoop/yarn/logs/application_1474152944798_0001/container_1474152944798_0001_02_000001
-Dtez.root.logger=WARN,CLA -Dsun.nio.ch.bugLevel=''
*org.apache.tez.dag.app.DAGAppMaster* --session
1>/storage5/hadoop/yarn/logs/application_1474152944798_0001/container_1474152944798_0001_02_000001/stdout
2>/storage5/hadoop/yarn/logs/application_1474152944798_0001/container_1474152944798_0001_02_000001/stderr
doesn't contain classnames [org.apache.hadoop.mapred.YarnChild,
org.apache.hadoop.mapreduce.v2.app.MRAppMaster] (Future similar messages
will be omitted up to 1000 times)

# and boom
2016-09-17 15:56:37,514 INFO
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor:
launchContainer: [bash,
/storage4/hadoop/yarn/local/usercache/dwr/appcache/application_1474152944798_0001/container_1474152944798_0001_02_000001/default_container_executor.sh]
2016-09-17 15:56:37,745 WARN
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit
code from container container_1474152944798_0001_02_000001 is : 1

# and from stderr from the datanode:

Error: Could not find or load main class org.apache.tez.dag.app.DAGAppMaster



so i ask why can't the container find DAGAppMaster from the tarball in
the first step?


Thanks,

Stephen.
PS I'm sure its something dumb but i just can't grok it.

Mime
View raw message