Hadoop recommendation for maximum open file descriptors

1.      Package Size and Inode Count Requirements

·         The recommended maximum number of open file descriptors is 10000 on each host, run below command to check:

ulimit -Sn

ulimit -Hn

·         If the output is not greater than 10000, run the following command to set it to a suitable default:

Edit /etc/sysctl.conf

fs.file-max = 65536

sysctl -p /etc/sysctl.conf

Edit /etc/security/limits.conf

Youruser or *       soft    nofile   10000

Youruser or *       hard    nofile   10000