| 1 |
NAME |
| 2 |
|
| 3 |
Hijk - Fast & minimal low-level HTTP client |
| 4 |
|
| 5 |
SYNOPSIS |
| 6 |
|
| 7 |
A simple GET request: |
| 8 |
|
| 9 |
use Hijk (); |
| 10 |
my $res = Hijk::request({ |
| 11 |
method => "GET", |
| 12 |
host => "example.com", |
| 13 |
port => "80", |
| 14 |
path => "/flower", |
| 15 |
query_string => "color=red" |
| 16 |
}); |
| 17 |
|
| 18 |
if (exists $res->{error} and $res->{error} & Hijk::Error::TIMEOUT) { |
| 19 |
die "Oh noes we had some sort of timeout"; |
| 20 |
} |
| 21 |
|
| 22 |
die "Expecting an 'OK' response" unless $res->{status} == 200; |
| 23 |
|
| 24 |
say $res->{body}; |
| 25 |
|
| 26 |
A POST request, you have to manually set the appropriate headers, URI |
| 27 |
escape your values etc. |
| 28 |
|
| 29 |
use Hijk (); |
| 30 |
use URI::Escape qw(uri_escape); |
| 31 |
|
| 32 |
my $res = Hijk::request({ |
| 33 |
method => "POST", |
| 34 |
host => "example.com", |
| 35 |
port => "80", |
| 36 |
path => "/new", |
| 37 |
head => [ "Content-Type" => "application/x-www-form-urlencoded" ], |
| 38 |
query_string => "type=flower&bucket=the%20one%20out%20back", |
| 39 |
body => "description=" . uri_escape("Another flower, let's hope it's exciting"), |
| 40 |
}); |
| 41 |
|
| 42 |
die "Expecting an 'OK' response" unless $res->{status} == 200; |
| 43 |
|
| 44 |
DESCRIPTION |
| 45 |
|
| 46 |
Hijk is a fast & minimal low-level HTTP client intended to be used |
| 47 |
where you control both the client and the server, e.g. for talking to |
| 48 |
some internal service from a frontend user-facing web application. |
| 49 |
|
| 50 |
It is NOT a general HTTP user agent, it doesn't support redirects, |
| 51 |
proxies, SSL and any number of other advanced HTTP features like (in |
| 52 |
roughly descending order of feature completeness) LWP::UserAgent, |
| 53 |
WWW::Curl, HTTP::Tiny, HTTP::Lite or Furl. This library is basically |
| 54 |
one step above manually talking HTTP over sockets. |
| 55 |
|
| 56 |
Having said that it's lightning fast and extensively used in production |
| 57 |
at Booking.com <https://www.booking.com> where it's used as the go-to |
| 58 |
transport layer for talking to internal services. It uses non-blocking |
| 59 |
sockets and correctly handles all combinations of connect/read timeouts |
| 60 |
and other issues you might encounter from various combinations of parts |
| 61 |
of your system going down or becoming otherwise unavailable. |
| 62 |
|
| 63 |
FUNCTION: Hijk::request( $args :HashRef ) :HashRef |
| 64 |
|
| 65 |
Hijk::request is the only function you should use. It (or anything else |
| 66 |
in this package for that matter) is not exported, so you have to use |
| 67 |
the fully qualified name. |
| 68 |
|
| 69 |
It takes a HashRef of arguments and either dies or returns a HashRef as |
| 70 |
a response. |
| 71 |
|
| 72 |
The HashRef argument to it must contain some of the key-value pairs |
| 73 |
from the following list. The value for host and port are mandatory, but |
| 74 |
others are optional with default values listed below. |
| 75 |
|
| 76 |
protocol => "HTTP/1.1", # (or "HTTP/1.0") |
| 77 |
host => ..., |
| 78 |
port => ..., |
| 79 |
connect_timeout => undef, |
| 80 |
read_timeout => undef, |
| 81 |
read_length => 10240, |
| 82 |
method => "GET", |
| 83 |
path => "/", |
| 84 |
query_string => "", |
| 85 |
head => [], |
| 86 |
body => "", |
| 87 |
socket_cache => \%Hijk::SOCKET_CACHE, # (undef to disable, or \my %your_socket_cache) |
| 88 |
on_connect => undef, # (or sub { ... }) |
| 89 |
parse_chunked => 0, |
| 90 |
head_as_array => 0, |
| 91 |
no_default_host_header => 1, |
| 92 |
|
| 93 |
Notice how Hijk does not take a full URI string as input, you have to |
| 94 |
specify the individual parts of the URL. Users who need to parse an |
| 95 |
existing URI string to produce a request should use the URI module to |
| 96 |
do so. |
| 97 |
|
| 98 |
The value of head is an ArrayRef of key-value pairs instead of a |
| 99 |
HashRef, this way you can decide in which order the headers are sent, |
| 100 |
and you can send the same header name multiple times. For example: |
| 101 |
|
| 102 |
head => [ |
| 103 |
"Content-Type" => "application/json", |
| 104 |
"X-Requested-With" => "Hijk", |
| 105 |
] |
| 106 |
|
| 107 |
Will produce these request headers: |
| 108 |
|
| 109 |
Content-Type: application/json |
| 110 |
X-Requested-With: Hijk |
| 111 |
|
| 112 |
In addition Hijk will provide a Host header for you by default with the |
| 113 |
host value you pass to request(). To suppress this (e.g. to send custom |
| 114 |
Host requests) pass a true value to the no_default_host_header option |
| 115 |
and provide your own Host header in the head ArrayRef (or don't, if you |
| 116 |
want to construct a Host-less request knock yourself out...). |
| 117 |
|
| 118 |
Hijk doesn't escape any values for you, it just passes them through |
| 119 |
as-is. You can easily produce invalid requests if e.g. any of these |
| 120 |
strings contain a newline, or aren't otherwise properly escaped. |
| 121 |
|
| 122 |
The value of connect_timeout or read_timeout is in floating point |
| 123 |
seconds, and is used as the time limit for connecting to the host, and |
| 124 |
reading the response back from it, respectively. The default value for |
| 125 |
both is undef, meaning no timeout limit. If you don't supply these |
| 126 |
timeouts and the host really is unreachable or slow, we'll reach the |
| 127 |
TCP timeout limit before returning some other error to you. |
| 128 |
|
| 129 |
The default protocol is HTTP/1.1, but you can also specify HTTP/1.0. |
| 130 |
The advantage of using HTTP/1.1 is support for keep-alive, which |
| 131 |
matters a lot in environments where the connection setup represents |
| 132 |
non-trivial overhead. Sometimes that overhead is negligible (e.g. on |
| 133 |
Linux talking to an nginx on the local network), and keeping open |
| 134 |
connections down and reducing complexity is more important, in those |
| 135 |
cases you can either use HTTP/1.0, or specify Connection: close in the |
| 136 |
request, but just using HTTP/1.0 is an easy way to accomplish the same |
| 137 |
thing. |
| 138 |
|
| 139 |
By default we will provide a socket_cache for you which is a global |
| 140 |
singleton that we maintain keyed on join($;, $$, $host, $port). |
| 141 |
Alternatively you can pass in socket_cache hash of your own which we'll |
| 142 |
use as the cache. To completely disable the cache pass in undef. |
| 143 |
|
| 144 |
The optional on_connect callback is intended to be used for you to |
| 145 |
figure out from production traffic what you should set the |
| 146 |
connect_timeout. I.e. you can start a timer when you call |
| 147 |
Hijk::request() that you end when on_connect is called, that's how long |
| 148 |
it took us to get a connection. If you start another timer in that |
| 149 |
callback that you end when Hijk::request() returns to you that'll give |
| 150 |
you how long it took to send/receive data after we constructed the |
| 151 |
socket, i.e. it'll help you to tweak your read_timeout. The on_connect |
| 152 |
callback is provided with no arguments, and is called in void context. |
| 153 |
|
| 154 |
We have experimental support for parsing chunked responses encoding. |
| 155 |
historically Hijk didn't support this at all and if you wanted to use |
| 156 |
it with e.g. nginx you had to add chunked_transfer_encoding off to the |
| 157 |
nginx config file. |
| 158 |
|
| 159 |
Since you may just want to do that instead of having Hijk do more work |
| 160 |
to parse this out with a more complex and experimental codepath you |
| 161 |
have to explicitly enable it with parse_chunked. Otherwise Hijk will |
| 162 |
die when it encounters chunked responses. The parse_chunked option may |
| 163 |
be turned on by default in the future. |
| 164 |
|
| 165 |
The return value is a HashRef representing a response. It contains the |
| 166 |
following key-value pairs. |
| 167 |
|
| 168 |
proto => :Str |
| 169 |
status => :StatusCode |
| 170 |
body => :Str |
| 171 |
head => :HashRef (or :ArrayRef with "head_as_array") |
| 172 |
error => :PositiveInt |
| 173 |
error_message => :Str |
| 174 |
errno_number => :Int |
| 175 |
errno_string => :Str |
| 176 |
|
| 177 |
For example, to send a request to http://example.com/flower?color=red, |
| 178 |
pass the following parameters: |
| 179 |
|
| 180 |
my $res = Hijk::request({ |
| 181 |
host => "example.com", |
| 182 |
port => "80", |
| 183 |
path => "/flower", |
| 184 |
query_string => "color=red" |
| 185 |
}); |
| 186 |
die "Response is not 'OK'" unless $res->{status} == 200; |
| 187 |
|
| 188 |
Notice that you do not need to put the leading "?" character in the |
| 189 |
query_string. You do, however, need to properly uri_escape the content |
| 190 |
of query_string. |
| 191 |
|
| 192 |
Again, Hijk doesn't escape any values for you, so these values MUST be |
| 193 |
properly escaped before being passed in, unless you want to issue |
| 194 |
invalid requests. |
| 195 |
|
| 196 |
By default the head in the response is a HashRef rather then an |
| 197 |
ArrayRef. This makes it easier to retrieve specific header fields, but |
| 198 |
it means that we'll clobber any duplicated header names with the most |
| 199 |
recently seen header value. To get the returned headers as an ArrayRef |
| 200 |
instead specify head_as_array. |
| 201 |
|
| 202 |
If you want to fiddle with the read_length value it controls how much |
| 203 |
we POSIX::read($fd, $buf, $read_length) at a time. |
| 204 |
|
| 205 |
We currently don't support servers returning a http body without an |
| 206 |
accompanying Content-Length header; bodies MUST have a Content-Length |
| 207 |
or we won't pick them up. |
| 208 |
|
| 209 |
ERROR CODES |
| 210 |
|
| 211 |
If we had a recoverable error we'll include an "error" key whose value |
| 212 |
is a bitfield that you can check against Hijk::Error::* constants. |
| 213 |
Those are: |
| 214 |
|
| 215 |
Hijk::Error::CONNECT_TIMEOUT |
| 216 |
Hijk::Error::READ_TIMEOUT |
| 217 |
Hijk::Error::TIMEOUT |
| 218 |
Hijk::Error::CANNOT_RESOLVE |
| 219 |
Hijk::Error::REQUEST_SELECT_ERROR |
| 220 |
Hijk::Error::REQUEST_WRITE_ERROR |
| 221 |
Hijk::Error::REQUEST_ERROR |
| 222 |
Hijk::Error::RESPONSE_READ_ERROR |
| 223 |
Hijk::Error::RESPONSE_BAD_READ_VALUE |
| 224 |
Hijk::Error::RESPONSE_ERROR |
| 225 |
|
| 226 |
In addition we might return error_message, errno_number and |
| 227 |
errno_string keys, see the discussion of Hijk::Error::REQUEST_* and |
| 228 |
Hijk::Error::RESPONSE_* errors below. |
| 229 |
|
| 230 |
The Hijk::Error::TIMEOUT constant is the same as |
| 231 |
Hijk::Error::CONNECT_TIMEOUT | Hijk::Error::READ_TIMEOUT. It's there |
| 232 |
for convenience so you can do: |
| 233 |
|
| 234 |
.. if exists $res->{error} and $res->{error} & Hijk::Error::TIMEOUT; |
| 235 |
|
| 236 |
Instead of the more verbose: |
| 237 |
|
| 238 |
.. if exists $res->{error} and $res->{error} & (Hijk::Error::CONNECT_TIMEOUT | Hijk::Error::READ_TIMEOUT) |
| 239 |
|
| 240 |
We'll return Hijk::Error::CANNOT_RESOLVE if we can't gethostbyname() |
| 241 |
the host you've provided. |
| 242 |
|
| 243 |
If we fail to do a select() or write() during when sending the response |
| 244 |
we'll return Hijk::Error::REQUEST_SELECT_ERROR or |
| 245 |
Hijk::Error::REQUEST_WRITE_ERROR, respectively. Similarly to |
| 246 |
Hijk::Error::TIMEOUT the Hijk::Error::REQUEST_ERROR constant is a union |
| 247 |
of these two, and any other request errors we might add in the future. |
| 248 |
|
| 249 |
When we're getting the response back we'll return |
| 250 |
Hijk::Error::RESPONSE_READ_ERROR when we can't read() the response, and |
| 251 |
Hijk::Error::RESPONSE_BAD_READ_VALUE when the value we got from read() |
| 252 |
is 0. The Hijk::Error::RESPONSE_ERROR constant is a union of these two |
| 253 |
and any other response errors we might add in the future. |
| 254 |
|
| 255 |
Some of these Hijk::Error::REQUEST_* and Hijk::Error::RESPONSE_* errors |
| 256 |
are re-thrown errors from system calls. In that case we'll also pass |
| 257 |
along error_message which is a short human readable error message about |
| 258 |
the error, as well as errno_number & errno_string, which are $!+0 and |
| 259 |
"$!" at the time we had the error. |
| 260 |
|
| 261 |
Hijk might encounter other errors during the course of the request and |
| 262 |
WILL call die if that happens, so if you don't want your program to |
| 263 |
stop when a request like that fails wrap it in eval. |
| 264 |
|
| 265 |
Having said that the point of the Hijk::Error::* interface is that all |
| 266 |
errors that happen during normal operation, i.e. making valid requests |
| 267 |
against servers where you can have issues like timeouts, network blips |
| 268 |
or the server thread on the other end being suddenly kill -9'd should |
| 269 |
be caught, categorized and returned in a structural way by Hijk. |
| 270 |
|
| 271 |
We're not currently aware of any issues that occur in such normal |
| 272 |
operations that aren't classified as a Hijk::Error::*, and if we find |
| 273 |
new issues that fit the criteria above we'll likely just make a new |
| 274 |
Hijk::Error::* for it. |
| 275 |
|
| 276 |
We're just not trying to guarantee that the library can never die, and |
| 277 |
aren't trying to catch truly exceptional issues like e.g. fcntl() |
| 278 |
failing on a valid socket. |
| 279 |
|
| 280 |
AUTHORS |
| 281 |
|
| 282 |
Kang-min Liu <[email protected]> |
| 283 |
|
| 284 |
Ãvar Arnfjörð Bjarmason <[email protected]> |
| 285 |
|
| 286 |
Borislav Nikolov <[email protected]> |
| 287 |
|
| 288 |
Damian Gryski <[email protected]> |
| 289 |
|
| 290 |
COPYRIGHT |
| 291 |
|
| 292 |
Copyright (c) 2013- Kang-min Liu <[email protected]>. |
| 293 |
|
| 294 |
LICENCE |
| 295 |
|
| 296 |
The MIT License |
| 297 |
|
| 298 |
DISCLAIMER OF WARRANTY |
| 299 |
|
| 300 |
BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY |
| 301 |
FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT |
| 302 |
WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER |
| 303 |
PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND, |
| 304 |
EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED |
| 305 |
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE |
| 306 |
ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH |
| 307 |
YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL |
| 308 |
NECESSARY SERVICING, REPAIR, OR CORRECTION. |
| 309 |
|
| 310 |
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING |
| 311 |
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR |
| 312 |
REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE |
| 313 |
TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR |
| 314 |
CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE |
| 315 |
SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING |
| 316 |
RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A |
| 317 |
FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF |
| 318 |
SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH |
| 319 |
DAMAGES. |
| 320 |
|