...
| Code Block | ||
|---|---|---|
| ||
<property>
<name<dfs<name>dfs.datanode.failed.volumes.tolerated</name>
<value>0</value>
</property> |
For those using Ambari, you can find the following UI snippet at Services > HDFS > Configs for this setting.
It is my personal recommendation that if your DataNodes are leveraging 4+ drives to set this value at approximately one-third of the number of disks as shown in the table below.
| Number of DN Drives | Drive Failure Toleranace |
|---|---|
| 4 | 1 |
| 6 | 2 |
| 8 | 2 |
| 10 | 3 |
| 12 | 4 |
The goal is surely not to try to run these worker nodes until the max value of the dfs.datanode.failed.volumes.tolerated property, but rather give the Hadoop administrator a chance to deal with this kind of hardware failure in an extremely non critical fashion. A single failure would be enough to identify the node to be pulled out in the next rotation which might be a weekly process. That process would also allow for a graceful decommission process to occur which would make sure all the necessary blocks are fully replicated before the DataNode process shuts down. Then the box can be repaired and can be brought back into the cluster.