How do I increase the open files limit for a non-root user?

  • This is happening on Ubuntu Release 12.04 (precise) 64-bit Kernel Linux 3.2.0-25-virtual

    I'm trying to increase the number of open files allowed for a user. This is for my eclipse java application where the current limit of 1024 is not enough.

    According to the posts I've found so far, I should be able to put lines into

    /etc/security/limits.conf like this:

    soft nofile 4096
    hard nofile 4096
    

    to increase the number of open files allowed for all users.

    But that's not working for me and I think the problem is not related to that file.

    For all users, the default limit is 1024, regardless of what is in /etc/security/limits.conf (I rebooted after changing that file)

    $ ulimit -n
    1024
    

    Now, despite the entries in /etc/security/limits.conf I can't increase that:

    $ ulimit -n 2048
    

    -bash: ulimit: open files: cannot modify limit: Operation not permitted The weird part is that I can change the limit downwards, but can't change it upwards - even to go back to a number which is below the original limit:

    $ ulimit -n 800
    $ ulimit -n
    800
    
    $ ulimit -n 900
    
    -bash: ulimit: open files: cannot modify limit: Operation not permitted
    

    As root, I can change that limit to whatever I want, up or down. It doesn't even seem to care about the supposedly system-wide limit in /proc/sys/fs/file-max

    # cat /proc/sys/fs/file-max
    188897
    
    # ulimit -n 188898
    # ulimit -n 
    188898
    

    But even I get eclipse to run as root, my application still crashes because of "Too Many Open File" exception!

    So far, I haven't found any way to increase the open files limit for a non-root user.

    How should I properly do this? I have looked at several other posts but no luck!

    Also, note that after you edit `/etc/security/limits.conf`, you may have to logout and then back in before you can use the new max limit. I did this, and was baffled by `ulimit -Hs` still showing 1000 when I had just raised it to 1000000! Then I logged out and back in, and ulimit showed the new amount.

    For GUI session you may find also important (Ubuntu 16+) this thread.

  • lornix

    lornix Correct answer

    8 years ago

    The ulimit command by default changes the HARD limits, which you (a user) can lower, but cannot raise.

    Use the -S option to change the SOFT limit, which can range from 0-{HARD}.

    I have actually aliased ulimit to ulimit -S, so it defaults to the soft limits all the time.

    alias ulimit='ulimit -S'
    

    As for your issue, you're missing a column in your entries in /etc/security/limits.conf.

    There should be FOUR columns, but the first is missing in your example.

    * soft nofile 4096
    * hard nofile 4096
    

    The first column describes WHO the limit is to apply for. '*' is a wildcard, meaning all users. To raise the limits for root, you have to explicitly enter 'root' instead of '*'.

    You also need to edit /etc/pam.d/common-session* and add the following line to the end:

    session required pam_limits.so
    

    How do I change the Hard limit for the maximum number of open file descriptors? My problem is that I need to allow my Eclipse application to have as many as open file as it wants but the limit of 4096 is not enough and I want to set it to as high as possible maybe 500000. My setting in the limits.conf is as you mentioned with the star and still the Hard limit is not moving.

    did you logout/reboot this time? `ulimit -a` will show you all the limits, so you can compare to see if they've changed.

    yes, I modified the /etc/security/limits.conf file only and reboot. after reboot, the ulimit -Hn500000 command still throws bash: ulimit: open files: cannot modify limit: Operation not permitted error.

    Well, 500,000 *IS* a bit large. I doubt you've got enough RAM to contain all the file descriptors for that many open files. Try something a bit smaller, say 65,000 or less. Although the docs say it should accept `unlimited` as a value too, also `infinity`, both should remove restrictions.

    If the value you supply is too large, it probably just ignores the setting.

    You are correct. I set it to 65000 and that worked but the issue is that I am still getting too many open file error! The library that I am using does this (does ask me why but it is valid) and I can not change it. What is the maximum number that I can set this number to?

    Well, you discovered the value of /proc/sys/fs/file-max was 188,897... So I'd say your upper limit is between 188,000 (allowed) and 500,000 (not allowed). Boggles the mind that you've got something that requires over 65000 open files simultaneously. Wow. Usually that's handled on big-iron. I'd be suspicious of a programming, uh, misconception. (With all due respect, please)

    Lovely - this worked like a charm.

    that was it: session required pam_limits.so, thanks!

    At first I missed the wildcard on the end of `/etc/pam.d/common-session*` and just edited `common-session`, and even after a reboot it didn't work. But after adding the same line (for `pam_limits.so`) to `common-session-noninteractive`, `ulimit -n` displayed the new value after a fresh login (no reboot required). FWIW I was trying to change the limit for root (only).

    I needed to add `session required pam_limits.so` to the `xrdp-sesman` file in order to have this word over RDP.

    I have the same issue if using root user...

    I followed the instructions in this post and now my virtual box virtual machine doesn't start up. I don't think it has to do with the fact that I applied the changes on my VM. Has this happened to anyone ?

  • If you use soft and hard limits on a per user basis you can use something like:

    su USER --shell /bin/bash --command "ulimit -n"
    

    to check wether your settings are working for that specific user or not.

    I was also able to use this to unzip a file against ulimit rules `su i --shell /bin/bash --command "ulimit -f 8000000 && gunzip -f file_name.gz"`

  • I have lots of trouble getting this to work.

    Using the following allows you to update it regardless of your user permission.

    sudo sysctl -w fs.inotify.max_user_watches=100000
    

    I tried this since I'm having same issue but it did not work `$ sudo sysctl -w fs.inotify.max_user_watches=100000 fs.inotify.max_user_watches = 100000 $ ulimit -n 1024`

  • ulimit -S -n 4096
    

    That should increase your soft limit for open files (-n) to 4096.

    The soft limit is rarely the problem, in my experience. It's the hard limit that has to be increased.

License under CC-BY-SA with attribution


Content dated before 6/26/2020 9:53 AM