Any way to encode the url in curl command?

  • I have some url which has space in it's query param. I want to use this in curl, e.g.

    curl -G "http://localhost:30001/data?zip=47401&utc_begin=2013-8-1 00:00:00&utc_end=2013-8-2 00:00:00&country_code=USA"
    

    which gives out

    Malformed Request-Line
    

    As per my understanding o/p is due to the space present in query param.

    Is there any away to encode the url automatically before providing it to curl command?

  • curl supports url-encoding internally with --data-urlencode:

    $ curl -G -v "http://localhost:30001/data" --data-urlencode "msg=hello world" --data-urlencode "msg2=hello world2"
    

    -G is also necessary to append the data to the URL.

    Trace headers

    > GET /data?msg=hello%20world&msg2=hello%20world2 HTTP/1.1
    > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu)
    > Host: localhost
    > Accept: */*
    

    What if `msg = '='` ?

    From curl doc: Note that the name part (msg in this case) is expected to be URL-encoded already. Also you can specify something like --request DELETE and it would indeed be a delete method instead of a GET. Not sure if order matters.

    @damphat what happens when the request has two parameters like `"msg1=Hello&msg2=World"`? This will encode the `&` between the parameters which would mean wrong thing to send to the server

    @GaneshSatpute: use multiple `--data-urlencode` parameters, one for each key-value pair.

    @MartijnPieters Yes. that works. Thanks for the answer

    @DimitriKopriwa just repeat the `--data-urlencode` option, like in the example provided

    Is it possible to perform query params url-encoding for a POST request **and** provide a payload ?

    i wonder why this answer is not accepted.

  •  curl -G "$( echo "$URL" | sed 's/ /%20/g' )"
    

    Where $URL is the url you want to do the translations on.

    There are also more than one type of translation (encoding) you can have in a URL, so you may want to do:

    curl -G "$(perl -MURI::Escape -e 'print uri_escape shift, , q{^A-Za-z0-9\-._~/:}' -- "$URL")"
    

    instead.

    Note that `echo "$URL" | sed 's/ /%20/'` won't do the right thing if there are `%` characters in the URL. Also, spaces are normally encoded as `+` (and `+` as `%2b`). I recommend the Perl solution, which is reliable.

    `sed 's/ /%20/g'` if you have more than one space to translate...

    Note I had to install the Perl URI::Escape module.

License under CC-BY-SA with attribution


Content dated before 6/26/2020 9:53 AM