Engineering, 07.03.2020 05:43 wbrandi118
JAVA HADOOP MAPREDUCE
Modify the WordCount program so it outputs the wordcount for each distinct word in each file. So the output of this DocWordCount program should be of the form ‘wordfilename count’, where ‘’ serves as a delimiter between word and filename and tab serves as a delimiter between filename and count. Submit your source code in a file named DocWordCount. java.
Explanation: Consider two simple files file1.txt and file2.txt. $ echo "Hadoop is yellow Hadoop" > file1.txt $ echo "yellow Hadoop is an elephant" > file2.txt Running ‘DocWordCount. java’ on these two files will give an output similar to that below, where is a delimiter.
Output of DocWordCount. java
yellowfile2.txt 1
Hadoopfile2.txt 1
isfile2.txt 1
elephantfile2.txt 1
yellowfile1.txt 1
Hadoopfile1.txt 2
isfile1.txt 1
anfile2.txt 1
Initial code that needs to be modified:
package org. myorg;
import java. io. IOException;
import java. util. regex. Pattern;
import org. apache. hadoop. conf. Configured;
import org. apache. hadoop. util. Tool;
import org. apache. hadoop. util. ToolRunner;
import org. apache. log4j. Logger;
import org. apache. hadoop. mapreduce. Job;
import org. apache. hadoop. mapreduce. Mapper;
import org. apache. hadoop. mapreduce. Reducer;
import org. apache. hadoop. fs. Path;
import org. apache. hadoop. mapreduce. lib. input. FileInputFormat;
import org. apache. hadoop. mapreduce. lib. output. FileOutputFormat;
import org. apache. hadoop. io. IntWritable;
import org. apache. hadoop. io. LongWritable;
import org. apache. hadoop. io. Text;
public class WordCount extends Configured implements Tool {
private static final Logger LOG = Logger .getLogger( WordCount. class);
public static void main( String[] args) throws Exception {
int res = ToolRunner .run( new WordCount(), args);
System .exit(res);
}
public int run( String[] args) throws Exception {
Job job = Job .getInstance(getConf(), " wordcount ");
job. setJarByClass( this .getClass());
FileInputFormat. addInputPaths(job, args[0]);
FileOutputFormat. setOutputPath(job, new Path(args[ 1]));
job. setMapperClass( Map .class);
job. setReducerClass( Reduce .class);
job. setOutputKeyClass( Text .class);
job. setOutputValueClass( IntWritable .class);
return job. waitForCompletion( true) ? 0 : 1;
}
public static class Map extends Mapper {
private final static IntWritable one = new IntWritable( 1);
private Text word = new Text();
private static final Pattern WORD_BOUNDARY = Pattern .compile("\\s*\\b\\s*");
public void map( LongWritable offset, Text lineText, Context context)
throws IOException, InterruptedException {
String line = lineText. toString();
Text currentWord = new Text();
for ( String word : WORD_BOUNDARY .split(line)) {
if (word. isEmpty()) {
continue;
}
currentWord = new Text(word);
context. write(currentWord, one);
}
}
}
public static class Reduce extends Reducer {
@Override
public void reduce( Text word, Iterable counts, Context context)
throws IOException, InterruptedException {
int sum = 0;
for ( IntWritable count : counts) {
sum += count. get();
}
context. write(word, new IntWritable(sum));
}
}
}
Answers: 1
Engineering, 03.07.2019 23:20
Two technicians are discussing the intake air temperature (iat) sensor. technician a says that the computer uses the iat sensor as a backup to the engine coolant temperature (ect) sensor. technician b says that the powertrain control module (pcm) will subtract the calculated amount of fuel if the air measures hot. who is correct
Answers: 3
Engineering, 04.07.2019 18:10
What difference(s) did you notice using a pneumatic circuit over hydraulic circuit.explain why the pneumatic piston stumbles when it hits an obstacle.
Answers: 2
Engineering, 04.07.2019 18:10
Ifa component is made of two or more materials with different modulus of elasticity (e), it is called a composite member and we calculate the factor·n". mention the formula for calculating n". also, ifn> 1, explain what will happen to the 1. transformed.gi) ifn 1, what will happen to the material when transformed material when
Answers: 1
Engineering, 04.07.2019 18:10
Compute the pressure drop of 30°c air flowing with a mean velocity of 8 m/s in a circular sheet-metal duct 300 mm in diameter and 15 m long. use a friction factor, f 0.02, and pair = 1.1644 kg/m a. 37.26 pa b. 25.27 pa n c. 29.34 pa d. 30.52 pa
Answers: 1
JAVA HADOOP MAPREDUCE
Modify the WordCount program so it outputs the wordcount for each...
Modify the WordCount program so it outputs the wordcount for each...
Mathematics, 12.01.2021 01:00
Mathematics, 12.01.2021 01:00
Mathematics, 12.01.2021 01:00
Mathematics, 12.01.2021 01:00
Mathematics, 12.01.2021 01:00
Mathematics, 12.01.2021 01:00
Mathematics, 12.01.2021 01:00
Chemistry, 12.01.2021 01:00
Mathematics, 12.01.2021 01:00
Physics, 12.01.2021 01:00