Instructions for use of LCSR's hadoop cluster

Instructions for use of LCSR's hadoop cluster

Setup

Usage

[Note: to be able to submit jobs to our hadoop cluster, a directory has to be created for you. Ask the CS professor sponsoring your work to request one.]

To submit (a sample) job:

    cd hadoop-0.20.203.0
    bin/hadoop jar hadoop-examples-*.jar grep input output 'dfs[a-z.]+'
One user reported an error trying to submit the example saying "Input path does not exist." He was able to fix this with the command
    bin/hadoop dfs -mkdir input
To review the output:
    bin/hadoop fs -get output output
    cat output/*

This page last updated May 18, 2012.