Saturday, February 28, 2015

Hadoop recommendation for maximum open file descriptors

1.      Package Size and Inode Count Requirements
·         The recommended maximum number of open file descriptors is 10000 on each host, run below command to check:
ulimit -Sn
ulimit -Hn
·         If the output is not greater than 10000, run the following command to set it to a suitable default:
Edit /etc/sysctl.conf
fs.file-max = 65536
sysctl -p /etc/sysctl.conf

Edit /etc/security/limits.conf
Youruser or *       soft    nofile   10000
Youruser or *       hard    nofile   10000