Set a network range in the no_proxy environment variable

  • I'm in a network using a proxy. I've got machines using lots of scripts here and there accessing each other over HTTP.

    • The network is 10.0.0.0/8.
    • My proxy is 10.1.1.1:81, so I set it up accordingly:

      export http_proxy=http://10.1.1.1:81/
      
    • I want to exclude my own range to be accessed with the proxy. I tried any combination available.

      export no_proxy='10.*'
      export no_proxy='10.*.*.*'
      export no_proxy='10.0.0.0/8'
      

    None of the above work!

    I'm testing with wget and it always tries to query the proxy, whatever IP address I want to connect to.

    • Since lots of scripts lie everywhere in all systems the --no-proxy option is actually not an option. I want to set it system wide.

    It's almost 2019 and no way to put CIDR in `no_proxy` environment variable? What the hell Linus Torvalds!!

    @7_R3X it's GNU wget so I expect you wanted to curse Richard Stallman?

    Richard Stallman has already been cursed for having a liberty to freely express own opinions. Curse Gnome.

  • janmoesen

    janmoesen Correct answer

    9 years ago

    You're looking at it the wrong way. The no_proxy environment variable lists the domain suffixes, not the prefixes. From the documentation:

    no_proxy: This variable should contain a comma-separated list of domain extensions proxy should not be used for.

    So for IPs, you have two options:

    1) Add each IP in full:

    printf -v no_proxy '%s,' 10.1.{1..255}.{1..255};
    export no_proxy="${no_proxy%,}";
    

    2) Rename wget to wget-original and write a wrapper script (called wget) that looks up the IP for the given URL's host, and determines if it should use the proxy or not:

    #!/bin/bash
    ip='';
    for arg; do
       # parse arg; if it's a URL, determine the IP address
    done;
    if [[ "$ip" =~ ^10\.1\. ]]; then
       wget-original --no-proxy "[email protected]";
    else
       wget-original "[email protected]";
    fi;
    

    Thank's a lot, I'm going to wrap my wget (and use hostnames instead of IP addresses).

    Will having 255*255 addresses in no_proxy risk performance issues?

    @dafrazzman: that could well be. After all, it's almost 800 kilobytes large, which is somewhat extreme for an environment variable. It also depends on how large your environment can be. For this extreme case, I would recommend the wrapper approach.

    Also, it should be `10.1.{0..255}.{0..255}`, or at least `10.1.{0..255}.{1..255}`. (I don't think the last bit can be 0 for "normal" devices, but I am not sure.)

    Do not add each IP in full - it will be too long and you won't be able to execute any commands in bash.

    NEVER DO THAT, this will generate something too long for any command, and, if you put this in ``/etc/environment`` file, this could break your server.

    This should be changed in wget. Other tools are able to parse the CIDR notation in no_proxy (curl, python, ruby, etc.).

License under CC-BY-SA with attribution


Content dated before 6/26/2020 9:53 AM