Hadoop: Specify YARN Queu for DistCP

On our cluster, we set up a dynamic resource pool.

Set the rules so that the first yarn will look at the specified queue, and then Check the username, then check the main group…

But using distcp I can’t seem to specify the queue, it just sets it as the main group.

This is how I run it now (no Works):

hadoop distcp -Dmapred.job.queue.name:root.default .......

You made a mistake in the parameter specification.

You should not use “:” to separate key/value Yes. You should use “=”.

The command should be

hadoop distcp -Dmapred.job.queue.name=root.default... ....

On our cluster, we set up a dynamic resource pool.

Set up rules so that the first yarn The line will look at the specified queue, then the username, and then the main group…

But using distcp I can’t seem to specify the queue, it just sets it as the main group.

This This is the way I run it now (doesn’t work):

hadoop distcp -Dmapred.job.queue.name:root.default .......

You made a mistake in the parameter specification.

You should not use “:” to separate key/value pairs. You should Use “=”.

The command should be

hadoop distcp -Dmapred.job.queue.name=root.default ....... 

Leave a Comment

Your email address will not be published.