The Oozie framework made by Yahoo and supported by Cloudera is designed to handle this task by representing each Hadoop job as a node in an directed acyclic graph.
Hadoopy with Oozie support. (using Python to write a Hadoop program)
http://brandynwhite.com/hadoopy-cython-based-mapreduce-library-for-py
Hadoop Taiwan User Group
2011年2月7日 星期一
installing and running OOzie
http://yahoo.github.com/oozie/installing.html
/ming2/git/oozie/bin$ ./mkdistro.sh
or
/ming2/git/oozie$ mvn install
/ming2/git/oozie/bin$ ./mkdistro.sh
or
/ming2/git/oozie$ mvn install
Build oozie 3.0.0 with Maven 3.0.0
[INFO] Installing /ming2/git/oozie/distro/pom.xml to /home/ming/.m2/repository/com/yahoo/oozie/oozie/3.0.0-SNAPSHOT/oozie-3.0.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Oozie Main ........................................ SUCCESS [3.464s]
[INFO] Oozie Client ...................................... SUCCESS [24.096s]
[INFO] Oozie Share Lib ................................... SUCCESS [12.913s]
[INFO] Oozie Core ........................................ SUCCESS [24:33.559s]
[INFO] Oozie Docs ........................................ SUCCESS [2.852s]
[INFO] Oozie WebApp ...................................... SUCCESS [2.430s]
[INFO] Oozie Examples .................................... SUCCESS [21.312s]
[INFO] Oozie Distro ...................................... SUCCESS [4:23.045s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:04.333s
[INFO] Finished at: Tue Feb 08 13:26:58 CST 2011
[INFO] Final Memory: 41M/299M
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Oozie Main ........................................ SUCCESS [3.464s]
[INFO] Oozie Client ...................................... SUCCESS [24.096s]
[INFO] Oozie Share Lib ................................... SUCCESS [12.913s]
[INFO] Oozie Core ........................................ SUCCESS [24:33.559s]
[INFO] Oozie Docs ........................................ SUCCESS [2.852s]
[INFO] Oozie WebApp ...................................... SUCCESS [2.430s]
[INFO] Oozie Examples .................................... SUCCESS [21.312s]
[INFO] Oozie Distro ...................................... SUCCESS [4:23.045s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:04.333s
[INFO] Finished at: Tue Feb 08 13:26:58 CST 2011
[INFO] Final Memory: 41M/299M
[INFO] ------------------------------------------------------------------------
訂閱:
意見 (Atom)