First, pass a cluster as a parameter Is it bad practice to give a function so that the function can schedule cluster usage among different sub-functions?
Secondly, I pass this cluster parameter to a function, so I think it will be the same cluster every time I use parLapply. Would it be better to use a new cluster for each parLapply call?
Thank you
RGDS
1) It is a good practice to pass the cluster as a function parameter. It is just a collection of connections to the work process, similar to the connection to a file .
2) There is no need to restart the cluster between calls. If a serious error occurs in a worker process, there will be a problem, but in this case, I recommend canceling the entire calculation and restarting the master Process.
I use the parLapply function on 20 cores. I think other functions such as parSapply are the same…
First of all, Is it bad practice to pass a cluster as a parameter to a function, so that the function can schedule the use of the cluster among different sub-functions?
Secondly, I pass this cluster parameter to a function, so I think it will be the same cluster every time I use parLapply. Would it be better to use a new cluster for each parLapply call?
Thank you
RGDS
I am not an expert in parallel computing, but I will take the risk anyway. < p>
1) It is a good practice to pass the cluster as a function parameter. It is just a connection collection of the worker process, similar to the connection to a file.
2) No need Restart the cluster between calls. If a serious error occurs in a worker process, there will be a problem, but in this case, I recommend canceling the entire calculation and restarting the main process.