In-mapper Combiner Program to Calculate Average
In my previous post I described how we can use in-mapper combiner to make our M/R program more efficient. In the post, we also saw both M/R algorithm for average calculation with/without using in-mapper combiner optimization.In this post I am posting codes for both the algorithm.
The M/R program to calculate average without in-mapper combiner is given below:
The M/R program to calculate average with in-mapper combiner is given below:
The programs took 56sec and 42sec respectively for execution on my laptop for 10million records. So we can see a 33% improvement in time while using in-mapper combiner program.
Hi please can you share the code of IntPair class?
ReplyDeleteCompletely forgot to write about it, just posted it here - http://rishavrohitblog.blogspot.com/2014/02/implementing-custom-writablecomparable.html
DeleteHi, your post on combiner programes were very easy to understand thanks for the useful post
ReplyDeleteHadoop Training in Velachery | Hadoop Training .
Hadoop Training in Chennai | Hadoop .
Nice post ! Thanks for sharing valuable information with us. Keep sharing..Big data hadoop online Training Course
ReplyDelete