Hadoop gen1 vs Hadoop gen2

I am a bit confused about the location of tasktracker in Hadoop-2.x.

The daemons in Hadoop-1.x are namenode, datanode, jobtracker, taskracker and secondarynamenode

The daemons in Hadoop-2.x are namenode, datanode, resourcemanager, applicationmaster, secondarynamenode.

This means that Jobtracker has been divided into: resourcemanager and applicationmaster

So where is tasktracker?

In YARN (the new execution framework in Hadoop 2), MapReduce does not exist as before.

YARN is a more general method of allocating resources on the cluster. ResourceManager, ApplicationMaster and NodeManager now contain the new YARN execution framework. NodeManager is a daemon on each node, so I Guess you can say that it replaced TaskTracker. But now it only provides processes instead of just mapping tasks and reducing tasks.

MapReduce still exists, but it is now an “application” of YARN.

The following is an introduction to YARN, which will be discussed in depth: http://hortonworks.com/blog/introducing-apache-hadoop-yarn/

I’m right I am a bit confused about the location of tasktracker in Hadoop-2.x.

The daemons in Hadoop-1.x are namenode, datanode, jobtracker, taskracker and secondarynamenode

Hadoop The daemons in -2.x are namenode, datanode, resourcemanager, applicationmaster, secondarynamenode.

This means that Jobtracker has been divided into: resourcemanager and applicationmaster

Where is the tasktracker?

In YARN (the new execution framework in Hadoop 2), MapReduce does not exist as before.

YARN is A more general method of allocating resources on the cluster. ResourceManager, ApplicationMaster, and NodeManager now contain the new YARN execution framework. NodeManager is a daemon on each node, so I guess you can say replace TaskTracker. But now it’s just Provide processes instead of just mapping tasks and reducing tasks.

MapReduce still exists, but it is now an “application” of YARN.

The following is an introduction to YARN, which will In-depth discussion: http://hortonworks.com/blog/introducing-apache-hadoop-yarn/

Leave a Comment

Your email address will not be published.