- Download
- 浏览: 233644 次
- 性别:
- 来自: 成都
文章分类
- 全部博客 (294)
- Hadoop (34)
- mysql (9)
- operatingsystem (13)
- Hive (8)
- Hue (8)
- Pig (11)
- oozie (8)
- ZooKeeper (1)
- HBase (1)
- Spark (4)
- Impala (1)
- Lily (0)
- Solr (41)
- RS (0)
- Sqoop (10)
- Avro (0)
- Thrift (0)
- HDP (0)
- Bigtop (0)
- Redis (7)
- Java (6)
- Tez (6)
- Ambari (1)
- Mahout (25)
- MongoDB (9)
- Lucene (9)
- Nutch (1)
- Katta (1)
- UIMA (0)
- MediaProcess (1)
- linux (1)
- Design (2)
- AI (1)
- RTR (1)
- Docker (1)
- Splunk (0)
- OpenNLP (1)
- Carrot (3)
- LingPipe (0)
- Weka (0)
- Hama (9)
- CloudStack (0)
- Helix (0)
- Rave (0)
- jclouds (0)
- Giraph (0)
- Drill (0)
- Tajo (0)
- Kafka (3)
- Samza (0)
- Storm (23)
- Flume (15)
- Sifarish (1)
- ML (3)
- android (2)
- Theory (2)
- 系统架构 (2)
- Kiji (1)
- Neo4j (16)
- spanner (0)
- Ejabberd (0)
- Dropwizard (1)
- Tigon (1)
- OrientDB (1)
- Kite (2)
- Jubatus (4)
- Logstash (2)
- Kibana (0)
- Cassandra (0)
- Curator (1)
最新评论
-
oldrat:
https://github.com/oldratlee/tr ...
Kafka: High Qulity Posts
- Build
According to official document, when run build command
$ bin/mkdistro.sh
there are some errors. By default it builds against Hadoop 1.1.1 and the build configuration enforces that JDK 1.6.* is being used to compile and run the testcases.
So do the following steps to fix the problem(You should replace 2.x.0 to your hadoop version)
#cd oozie-4.0.0/
#find . -name pom.xml | xargs sed -ri 's/(2.2.0\-SNAPSHOT)/2.x.0/'
//or you should alter pom.xml manually
//java 1.6
mvn -DskipTests=true -P hadoop-2 clean package assembly:single
//java 1.7
$ mvn clean package assembly:single -Dhadoop.version=2.x.0 \
-DjavaVersion=1.7 -DtargetJavaVersion=1.7 -DskipTests
-Dpig.version=0.12.0 \
-DgenerateDocs
Note: I can build oozie4.0.1 succesfully by the following steps
#tar -xzf oozie-4.0.1.tar.gz
#cd oozie-4.0.1
#mvn clean package assembly:single -Dhadoop.version=2.3.0 \
-DjavaVersion=1.7 -DtargetJavaVersion=1.7 -DskipTests -Dpig.version=0.12.0 \
-DgenerateDocs
You should change your system language to english, otherwise some pom or jar cannot download from
couldera site.
Server Installation
#cd oozie-4.0.1/distro/target
#cp oozie-4.0.1-distro.tar.gz /path/to/oozie-install-dir
#cp oozie-4.0.1/hadooplibs/target/oozie-4.0.1-hadooplibs.tar.gz /path/to/oozie-install-dir
#cd /path/to/oozie-install-dir
#tar -xzf oozie-4.0.1-distro.tar.gz
#tar -xzf oozie-4.0.1-hadooplibs.tar.gz
#cd oozie-4.0.1/
#mkdir libext
#cp hadooplibs/hadooplib-2.3.0.oozie-4.0.1/* libext/
#cp /path/to/ext-2.2.zip libext/
#cp /path/to/mysql-connector-java-5.1.29-bin.jar libext/
#tree -L 1
#bin/oozie-setup.sh prepare-war #bin/oozie-setup.sh sharelib create -fs hdfs://192.168.122.1:2014 #bin/oozie-setup.sh db create -run
Expand the oozie-sharelib TAR.GZ file bundled with the distribution.
The share/ directory must be copied to the Oozie HOME directory in HDFS:
$ hadoop fs -put share share
IMPORTANT: This must be done using the Oozie Hadoop (HDFS) user. If a share directory already exists in HDFS, it must be deleted before copying it again.
$ bin/oozied.sh start
$ bin/oozied.sh run
$ bin/oozie admin -oozie http://localhost:11000/oozie -status
http://zhj-c0:11000/oozie/
Client Installation
expand the oozie-client TAR.GZ file bundled with the distribution. Add the bin/ directory to the PATH .
#oozie
ref:
https://oozie.apache.org/docs/4.0.1/DG_QuickStart.html
http://oozie.apache.org/docs/4.0.0/DG_QuickStart.html
https://oozie.apache.org/docs/4.0.0/ENG_Building.html
http://archive.cloudera.com/cdh/3/oozie/AG_Install.html
http://stackoverflow.com/questions/22077120/install-oozie-on-hadoop-2-2
grepalex.com/2014/02/16/oozie-and-hadoop-2.2/
http://mail-archives.apache.org/mod_mbox/oozie-user/201312.mbox/%3CCEDDEB80.76815%25chitnis@yahoo-inc.com%3E
http://www.hadoopinrealworld.com/building-running-and-testing-apache-oozie-4-0-0-2/
https://issues.apache.org/jira/browse/OOZIE-1551
https://issues.apache.org/jira/
发表评论
-
Add third party jars in a job
2014-08-18 15:10 1299When I submit a java job (inc ... -
Oozie: sqoop 1.4.4 sharelib
2014-04-11 13:31 779ENV : oozie4.0.1 hadoop2.3.0 ... -
Oozie: Using Mysql to store oozie metadata
2014-04-04 14:50 6061. configure oozie-site.xml ... -
Oozie: configuration
2014-04-04 12:42 685---conf/oozie-site.xml--- ... -
Oozie: Run examples
2014-04-04 12:43 816#cd /path/to/oozie-4.0.1 #t ... -
oozie: common errors
2014-04-04 10:41 18351. when run oozie examples, t ... -
Oozie: Sqoop Action Extension
2014-04-03 16:04 0Refere ... -
oozie:Expression Language Functions
2014-04-03 15:50 0... -
oozie: Workflow
2014-04-03 16:30 919Workflow Definition A workfl ... -
oozie:Command Line Interface Utilities
2014-04-03 14:16 0Refere ...
相关推荐
oozie-5.1.0.tar.gz 编译结果,受限上传大小,文件分3部分
oozie-4.1源码。github下载的。 oozie-4.1源码。github下载的。
2019-06-25 最新oozie5.0.0.tar.gz基于工作流调度hadoop作业web工具
oozie-4.3.1.tar.gz 源码,可以利用该tar包进行编译安装oozie
Oozie由Cloudera公司贡献给Apache的基于工作流引擎的开源框架,是用于Hadoop平台的开源的工作流调度引擎,是用来管理Hadoop作业,属于web应用程序,
大数据相关,大数据资源调度,oozie,可创建定时调度任务
By the end of the book, you will have a good knowledge of Apache Oozie. You will be capable of using Oozie to handle large Hadoop workflows and even improve the availability of your Hadoop environment...
linux.64 下的 oozie-4.3.0.tar.gz 源码包 解压后编译即可 注意对应的版本 cd bin ./mkdistro.sh -Phadoop-2 -Dhadoop.auth.version=2.8.2 -Ddistcp.version=2.8.2 -Dhadoop.version=2.8.2 -Dsqoop.version=1.4.6 -...
oozie-5.1.0.tar.gz 编译结果,受限上传大小,文件分3部分
With this book, you will be able to make your life easier by using tools such as Apache Hive, Apache Oozie, Hadoop Streaming, Apache Crunch, and Kite SDK. The last part of this book discusses the ...
oozie 4.1.0 linux安装包
Oozie由Cloudera公司贡献给Apache的基于工作流引擎的开源框架,是用于Hadoop平台的开源的工作流调度引擎,是用来管理Hadoop作业,属于web应用程序,由Oozie client和Oozie Server两个组件构成,Oozie Server运行于Java ...
适用于Hadoop 2.x的Oozie 这是一个映像,该映像对oozie / webapp的uber配置文件进行了一些更改,并使用hadoop-2配置文件和Hadoop 2.7.0库构建了一个Oozie发行版。 用法 将Oozie sharelib安装到HDFS docker run -ti...
dashuju 大数据oozie尚硅谷主讲,内容总结详细
oozie-5.1.0.tar.gz 编译结果,受限上传大小,文件分3部分
大数据Oozie架构原理.pdf
Giraph, GraphX, and other tools for large graph processing on Hadoop Using workflow orchestration and scheduling tools such as Apache Oozie Near-real-time stream processing with Apache Storm, Apache ...
ambari2.7.x源码编译需要使用到的hadoop对应包
2.Oozie的功能及模块 3.Oozie的与MR的关系 4.Oozie的工作流实现原理 5.Oozie中的特殊概念Action 6.Hadoop的集成配置 7.Oozie的部署配置 第三章:Oozie调度开发实战 1.Shell脚本工作流调度开发实战 2.Hive...
第3章 Oozie的部署 3.1 部署Hadoop(CDH版本的) 3.1.2 修改Hadoop配置 core-site.xml hadoop.proxyuser.atguigu.hosts * hadoop.proxyuser.atguigu.groups * mapred-site.xml mapreduce.jobhistory....