Limit memory usage for a single Linux process

  • I'm running pdftoppm to convert a user-provided PDF into a 300DPI image. This works great, except if the user provides an PDF with a very large page size. pdftoppm will allocate enough memory to hold a 300DPI image of that size in memory, which for a 100 inch square page is 100*300 * 100*300 * 4 bytes per pixel = 3.5GB. A malicious user could just give me a silly-large PDF and cause all kinds of problems.

    So what I'd like to do is put some kind of hard limit on memory usage for a child process I'm about to run--just have the process die if it tries to allocate more than, say, 500MB of memory. Is that possible?

    I don't think ulimit can be used for this, but is there a one-process equivalent?

  • kvz

    kvz Correct answer

    9 years ago

    There's some problems with ulimit. Here's a useful read on the topic: Limiting time and memory consumption of a program in Linux, which lead to the timeout tool, which lets you cage a process (and its forks) by time or memory consumption.

    The timeout tool requires Perl 5+ and the /proc filesystem mounted. After that you copy the tool to e.g. /usr/local/bin like so:

    curl https://raw.githubusercontent.com/pshved/timeout/master/timeout | \
      sudo tee /usr/local/bin/timeout && sudo chmod 755 /usr/local/bin/timeout
    

    After that, you can 'cage' your process by memory consumption as in your question like so:

    timeout -m 500 pdftoppm Sample.pdf
    

    Alternatively you could use -t <seconds> and -x <hertz> to respectively limit the process by time or CPU constraints.

    The way this tool works is by checking multiple times per second if the spawned process has not oversubscribed its set boundaries. This means there actually is a small window where a process could potentially be oversubscribing before timeout notices and kills the process.

    A more correct approach would hence likely involve cgroups, but that is much more involved to set up, even if you'd use Docker or runC, which among things, offer a more user-friendly abstraction around cgroups.

    Seems to be working for me now (again?) but here's the google cache version: http://webcache.googleusercontent.com/search?q=cache:zfQQHbZqcZQJ:coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/40+&;cd=1&hl=en&ct=clnk&gl=us

    Can we use timeout together with taskset (we need to limit both memory and cores) ?

    It should be noted that this answer is not referring to the linux standard `coreutils` utility of the same name! Thus, the answer is potentially dangerous if anywhere on your system, some package has a script expecting `timeout` to be the linux standard `coreutils` package! I am unaware of this tool being packaged for distributions such as debian.

    Does `-t ` constraint kill the process after that many seconds?

    Might also be helpful that -m is accepting kilobytes. The example above suggests its using MB.

License under CC-BY-SA with attribution


Content dated before 6/26/2020 9:53 AM