|
- .\" .IX httperf
- .TH "httperf" "1" "01 Feb 2008" "" ""
- .SH "NAME"
- httperf \- HTTP performance measurement tool
- .SH "SYNOPSIS"
- .B httperf
- .RB [ \-\-add\-header
- .I R S ]
- .RB [ \-\-burst\-length
- .I R N ]
- .RB [ \-\-client
- .I R I / N ]
- .RB [ \-\-close\-with\-reset ]
- .RB [ \-d | \-\-debug
- .I R N ]
- .RB [ \-\-failure\-status
- .I R N ]
- .RB [ \-h | \-\-help ]
- .RB [ \-\-hog ]
- .RB [ \-\-http\-version
- .I R S ]
- .RB [ \-\-max\-connections
- .I R N ]
- .RB [ \-\-max\-piped\-calls
- .I R N ]
- .RB [ \-\-method
- .I R S ]
- .RB [ \-\-no\-host\-hdr ]
- .RB [ \-\-num\-calls
- .I R N ]
- .RB [ \-\-num\-conns
- .I R N ]
- .RB [ \-\-period " [" d | u | e ] \fIT1\fR [ ,\fIT2\fR ]]
- .RB [ \-\-port
- .I R N ]
- .RB [ \-\-print\-reply " [" header | body ] ]
- .RB [ \-\-print\-request " [" header | body ] ]
- .RB [ \-\-rate
- .I R X ]
- .RB [ \-\-recv\-buffer
- .I R N ]
- .RB [ \-\-retry\-on\-failure ]
- .RB [ \-\-send\-buffer
- .I R N ]
- .RB [ \-\-server
- .I R S ]
- .RB [ \-\-server\-name
- .I R S ]
- .RB [ \-\-session\-cookie ]
- .RB [ \-\-ssl ]
- .RB [ \-\-ssl\-ciphers
- .I R L ]
- .RB [ \-\-ssl\-no\-reuse ]
- .RB [ \-\-think\-timeout
- .I R X ]
- .RB [ \-\-timeout
- .I R X ]
- .RB [ \-\-uri
- .I R S ]
- .RB [ \-v | \-\-verbose ]
- .RB [ \-V | \-\-version ]
- .RB [ "\-\-wlog y" | n, \fIF\fR]
- .RB [ \-\-wsess
- .I R N , N , X ]
- .RB [ \-\-wsesslog
- .I R N , X , F ]
- .RB [ \-\-wset
- .I R N , X ]
- .SH "DESCRIPTION"
- .B httperf
- is a tool to measure web server performance. It speaks the HTTP
- protocol both in its HTTP/1.0 and HTTP/1.1 flavors and offers a
- variety of workload generators. While running, it keeps track of a
- number of performance metrics that are summarized in the form of
- statistics that are printed at the end of a test run. The most basic
- operation of
- .B httperf
- is to generate a fixed number of HTTP GET requests and to measure how
- many replies (responses) came back from the server and at what rate
- the responses arrived.
-
- .B IMPORTANT:
- To obtain correct results, it is necessary to run at most one
- .B httperf
- process per client machine. Also, there should be as few background
- processes as possible both on the client and server machines.
-
- .SH "EXAMPLES"
- .TP
- httperf \-\-hog \-\-server www
- This command causes
- .B httperf
- to create a connection to host www, send a request for the root
- document (http://www/), receive the reply, close the connection, and
- then print some performance statistics.
- .TP
- httperf \-\-hog \-\-server www \-\-num\-conn 100 \-\-ra 10 \-\-timeout 5
- Like above, except that a total of 100 connections are created and
- that connections are created at a fixed rate of 10 per second. Note
- that option ``\-\-rate'' has been abbreviated to ``\-\-ra''.
- .TP
- httperf \-\-hog \-\-server=www \-\-wsess=10,5,2 \-\-rate 1 \-\-timeout 5
- Causes
- .B httperf
- to generate a total of 10 sessions at a rate of 1 session per second.
- Each session consists of 5 calls that are spaced out by 2 seconds.
- .TP
- httperf \-\-hog \-\-server=www \-\-wsess=10,5,2 \-\-rate=1 \-\-timeout=5 \-\-ssl
- Like above, except that
- .B httperf
- contacts server www via SSL at port 443 (the default port for SSL
- connections).
- .TP
- httperf \-\-hog \-\-server www \-\-wsess=10,5,2 \-\-rate=1 \-\-timeout=5 \-\-ssl \-\-ssl\-ciphers=EXP\-RC4\-MD5:EXP\-RC2\-CBC\-MD5 \-\-ssl\-no\-reuse \-\-http\-version=1.0
- Like above, except that
- .B httperf
- will inform the server that it can only select from two cipher suites
- (EXP\-RC4\-MD5 or EXP\-RC2\-CBC\-MD5); furthermore,
- .B httperf
- will use HTTP version 1.0 which requires a new TCP connection for each
- request. Also, SSL session ids are not reused, so the entire SSL
- connection establishment process (known as the SSL handshake) occurs
- for each connection.
- .SH "OPTIONS"
- The operation of
- .B httperf
- can be controlled through a number of options. The tool supports both
- short (one\-character) and long (arbitrary\-length) option names. Short
- options are prefixed with a single leading dash (\-), long
- options with a double\-dash (\-\-). Multiple short options can be
- grouped together (e.g.,
- .RB `` \-vV ''
- is equivalent to
- .RB `` "\-v \-V" '')
- and long options can be abbreviated so long as they remain unique.
- Parameters to options can be specified either by following the long
- option name with an equal sign and the parameter value (e.g.,
- .BR \-\-burst=10 )
- or by separating the option name and value with whitespace (e.g.,
- .BR "\-\-burst 10" ).
- .TP
- .BI \-\-add\-header= S
- Specifies to include string
- .I S
- as an additional request header. It is necessary to specify the
- terminating carriage\-return/line\-feed sequence explicitly. This can
- be done by using the escape sequence ``\\n''. This makes it possible
- to include multiple request headers. For example, ``\-\-add\-header
- "Referer: foo\\nAuth: secret\\n"'' would add two request headers
- (``Referer'' and ``Auth'') to each request. Other supported escape
- sequences are ``\\r'' (carriage\-return), ``\\a'' (line\-feed), ``\\\\''
- (backslash), and ``\\N'' where N is the code the character to be
- inserted (in octal).
- .TP
- .BI \-\-burst\-length= N
- Specifies the length of bursts. Each burst consists of
- .I N
- calls to the server. The exact meaning of this parameter depends on
- the workload generator. For regular request\-oriented workloads, see the
- description of option
- .BR \-\-wsess .
- .TP
- .BR \-\-no\-host\-hdr
- Specifies that the "Host:" header should not be included when issuing
- an HTTP request.
- .TP
- .BR \-\-num\-calls .
- For session\-oriented workloads, see the description of option
- .BR \-\-wsess .
- .TP
- .BI \-\-client= I / N
- Specifies that the machine
- .B httperf
- is running on is client
- .I I
- out of a total of
- .I N
- clients.
- .I I
- should be in the range from 0 to
- .I R N "\-1."
- Some of the workload generators (e.g.,
- .BR \-\-wset)
- use the client identity as a bias value to ensure that not all clients
- generate perfectly identical workloads. When performing a test that
- involves several client machines, it is generally a good idea to
- specify this option.
- .TP
- .BI \-\-close\-with\-reset
- Requests that
- .B httperf
- closes TCP connections by sending a RESET instead of going through the
- normal TCP connection shutdown handshake. Turning on this option can
- have ill effects such as data corruption, stuck TCP control blocks, or
- wrong results. For this reason, the option should not be used unless
- absolutely necessary and even then it should not be used unless its
- implications are fully understood.
- .TP
- .BI \-d= N
- .TP
- .BI \-\-debug= N
- Set debug level to
- .I R N .
- Larger values of
- .I N
- will result in more output.
- .TP
- .BI \-\-failure\-status= N
- Specifies that an HTTP response status code of
- .I N
- should be treated as a failure (i.e., treated as if the request had
- timed out, for example). For example, with
- .RB `` \-\-failure\-status=504 ''
- responses with an HTTP status of ``504 Gateway Time\-out'' would be
- considered failures. Caveat: this option is currently supported
- for session workloads only (see the
- .B \-\-wsess
- and
- .B \-\-wsesslog
- options).
- .TP
- .B \-h
- .TP
- .B \-\-help
- Prints a summary of available options and their parameters.
- .TP
- .BI \-\-hog
- This option requests to use up as many TCP ports as necessary.
- Without this option,
- .B httperf
- is typically limited to using ephemeral ports (in the range from 1024
- to 5000). This limited port range can quickly become a bottleneck so
- it is generally a good idea to specify this option for serious
- testing. Also, this option must be specified when measuring NT
- servers since it avoids a TCP incompatibility between NT and UNIX
- machines.
- .TP
- .BI \-\-http\-version= S
- Specifies the version string that should be included in the requests
- sent to the server. By default, version string ``1.1'' is used. This
- option can be set to ``1.0'' to force the generation of HTTP/1.0
- requests. Setting this option to any value other than ``1.0'' or ``1.1''
- may result in undefined behavior.
- .TP
- .BI \-\-max\-connections= N
- Specifies that at most
- .I N
- connections are opened for each session. This option is meaningful in
- conjunction with options
- .B \-\-wsess
- and
- .B \-\-wsesslog
- only.
- .TP
- .BI \-\-max\-piped\-calls= N
- Specifies that at most
- .I N
- pipelined calls are issued on each connection. This option is
- meaningful in conjunction with options
- .B \-\-wsess
- and
- .B \-\-wsesslog
- only.
- .TP
- .BI \-\-method= S
- Specifies the method that should be used when issuing an HTTP request.
- If this option is not specified, the GET method is used. The method
- .I S
- can be an arbitrary string but is usually one of GET, HEAD, PUT, POST,
- etc.
- .TP
- .BI \-\-num\-calls= N
- This option is meaningful for request\-oriented workloads only. It
- specifies the total number of calls to issue on each connection before
- closing it. If
- .I N
- is greater than 1, the server must support persistent connections.
- The default value for this option is 1. If
- .BR \-\-burst\-length
- is set to
- .I R B ,
- then the
- .I N
- calls are issued in bursts of
- .I B
- pipelined calls each. Thus, the total number of such bursts will
- be
- .I N/B
- (per connection).
- .TP
- .BI \-\-num\-conns= N
- This option is meaningful for request\-oriented workloads only. It
- specifies the total number of connections to create. On each
- connection, calls are issued as specified by options
- .B \-\-num\-calls
- and
- .BR \-\-burst\-length .
- A test stops as soon as the
- .I N
- connections have either completed or failed. A connection is
- considered to have failed if any activity on the connection fails to
- make forward progress for more than the time specified by the timeout
- options
- .B \-\-timeout
- and
- .BR \-\-think\-timeout .
- The default value for this option is 1.
- .TP
- .BI \-\-period= [D]T1[,T2]
- Specifies the time interval between the creation of connections or sessions.
- Connections are created by default, sessions if option
- .B \-\-wsess
- or
- .B \-\-wsesslog
- has been specified.
- This connection/session ``interarrival time'' can alternatively be specified by
- the
- .B \-\-rate
- option, although more flexibility is available with
- .B \-\-period.
- The
- .I D
- parameter specifies the interarrival time distribution.
- If omitted or set to
- .RB `` d '',
- a deterministic (i.e., fixed) period is used as specified
- by parameter
- .I R T1
- in units of seconds.
- If
- .I D
- is set to
- .RB `` e '',
- an exponential (i.e., Poisson) distribution is used with
- a mean interarrival time of
- .I R T1 .
- If
- .I D
- is set to
- .RB `` u '',
- a uniform distribution over the interval
- .RI [ T1 , T2 )
- is used for the interarrival time.
- Finally, if
- .I D
- is set to
- .RB ``v'',
- a number of rates can be specified as follows:
- .B \-\-period=vT1,D1,T2,D2...Tn,Dn
- Where n <= NUM_RATES in httperf.h and
- .I Ti,Di
- represent the period time (i.e., 1/rate) and duration to
- maintain that rate (i.e.,
- .B \-\-period=v1,2,0.5,4
- will generate 1 request/seconds for 2 seconds then
- 2 requests/seconds for 4 seconds).
- In all cases, a period of 0 results in connections
- or sessions being generated sequentially (a new connection/session is
- initiated as soon as the previous one completes). The default value
- for this option is 0. Note that specifying, for example,
- .B \-\-rate=5
- is equivalent to specifying
- .B \-\-period=d0.2
- or
- .BR \-\-period=0.2 .
- By specifying
- .BR \-\-period=u1,3 ,
- the interarrival times will be randomly chosen from the interval
- between 1 and 3 seconds. The specific sequence of (pseudo\-)random
- interarrival times are identical from one
- .B httperf
- run to another as long as the values for the
- .B \-\-period
- and
- .B \-\-client
- options are identical.
- .TP
- .BI \-\-port= N
- This option specifies the port number
- .I N
- on which the web server is listening for HTTP requests. By default,
- .B httperf
- uses port number 80.
- .TP
- .BR \-\-print\-reply [ = [ header | body ]]
- Requests the printing of the reply headers, body, and summary. The
- output is directed to standard output. Reply header lines are
- prefixed by "RH", reply body lines are prefixed by "RB", and the
- reply\-size summary is prefixed by "RS". The prefix is followed by a
- serial number that uniquely identifies the call that the reply line is
- for and a colon (":") character that marks the beginning of the actual
- reply line. To print only reply headers, pass argument
- .B header
- to this option. To print only the reply body, pass argument
- .B body
- to this option.
- .TP
- .BR \-\-print\-request [ = [ header | body ]]
- Requests the printing of the request headers, body (if one is
- present), and summary. The output is directed to standard output.
- Request header lines are prefixed by "SH", request body lines are
- prefixed by "SB", and the request summary is prefixed by "SS". The
- prefix is followed by the call's serial number and a colon (":")
- character that marks the beginning of the actual reply line. To print
- only request headers, pass argument
- .B header
- to this option. To print only the request body, pass argument
- .B body
- to this option.
- .TP
- .BI \-\-rate= X
- Specifies the fixed rate at which connections or sessions are created.
- Connections are created by default, sessions if option
- .B \-\-wsess
- or
- .B \-\-wsesslog
- has been specified. In both cases a rate of 0 results in connections
- or sessions being generated sequentially (a new session/connection is
- initiated as soon as the previous one completes). The default value
- for this option is 0.
- .TP
- .BI \-\-recv\-buffer= N
- Specifies the maximum size of the socket receive buffers used to
- receive HTTP replies. By default, the limit is 16KB. A smaller value
- may help memory\-constrained clients whereas a larger value may be
- necessary when communicating with a server over a high\-bandwidth,
- high\-latency connection.
- .TP
- .BI \-\-retry\-on\-failure
- This option is meaningful for session workloads only (see the
- .B \-\-wsess
- and
- .B \-\-wsesslog
- options). If specified, a call that results in a failure response (as
- defined by the
- .B \-\-failure\-status
- option) is retried immediately instead of causing the session to fail.
- .TP
- .BI \-\-send\-buffer= N
- Specifies the maximum size of the socket send buffers used to send
- HTTP requests. By default, the limit is 4KB. A smaller value may
- help memory\-constrained clients whereas a larger value may be
- necessary when generating large requests to a server connected via a
- high\-bandwidth, high\-latency connection.
- .TP
- .BI \-\-server= S
- Specifies the IP hostname of the server. By default, the hostname
- ``localhost'' is used. This option should always be specified as it
- is generally not a good idea to run the client and the server on the
- same machine.
- .TP
- .BI \-\-server\-name= S
- Specifies the (default) server name that appears in the "Host:" header
- of every request sent by
- .BR httperf .
- Without this option, the host name (or IP address) specified by option
- .B \-\-server
- is used instead.
- .TP
- .B \-\-session\-cookie
- When this option is turned on, cookie managment is enabled on a
- per\-session basis. What this means is that if a reply to a request
- that was generated by session
- .I R X
- contains a cookie, then all future requests sent by session
- .I X
- will include this cookie as well. At present, the cookie manager in
- .B httperf
- supports only one cookie per session. If a second cookie is received,
- the new cookie overwrites the existing one and a warning message is
- printed if ``\-\-debug 1'' is on.
- .TP
- .B \-\-ssl
- Specifies that all communication between
- .B httperf
- and the server should utilize the Secure Sockets Layer (SSL) protocol.
- This option is available only if
- .B httperf
- was compiled with SSL support enabled.
- .TP
- .BI \-\-ssl\-ciphers= L
- This option is only meaningful if SSL is in use (see
- .B \-\-ssl
- option). This option specifies the list
- .I L
- of cipher suites that
- .B httperf
- may use in negotiating a secure connection with the server. If the
- list contains more than one cipher suite, the ciphers must be
- separated by a colon. If the server does not accept any of the listed
- cipher suites, the connection establishment will fail and
- .B httperf
- will exit immediately. If this option is not specified when the
- .B \-\-ssl
- option is present then
- .B httperf
- will use all of the SSLv3 cipher suites provided by the underlying SSL
- library.
- .TP
- .B \-\-ssl\-no\-reuse
- This option is only meaningful if SSL and sessions are in use (see
- .BR \-\-ssl ,
- .BR \-\-wsess ,
- .BR \-\-wsesslog ).
- When an SSL connection is established the client receives a session
- identifier (session id) from the server. On subsequent SSL
- connections, the client normally reuses this session id in order to
- avoid the expense of repeating the (slow) SSL handshake to establish a
- new SSL session and obtain another session id (even if the client
- attempts to re\-use a session id, the server may force the client to
- renegotiate a session). By default
- .B httperf
- reuses the session id across all connections in a session. If the
- .B \-\-ssl\-no\-reuse
- option is in effect, then
- .B httperf
- will not reuse the session id, and the entire SSL handshake will be
- performed for each new connection in a session.
- .TP
- .BI \-\-think\-timeout= X
- Specifies the maximum time that the server may need to initiate
- sending the reply for a given request. Note that this timeout value
- is added to the normal timeout value (see option
- .BR \-\-timeout ).
- When accessing static web content, it is usually not necessary to
- specify this option. However, when performing tests with long\-running
- CGI scripts, it may be necessary to use this option to allow for
- larger response\-times. The default value for this option is zero
- seconds, meaning that the server has to be able to respond within the
- normal timeout value.
- .TP
- .BI \-\-timeout= X
- Specifies the amount of time
- .I X
- that
- .B httperf
- is willing to wait for a server reaction. The timeout is specified in
- seconds and can be a fractional number (e.g.,
- .BR "\-\-timeout 3.5" ).
- This timeout value is used when establishing a TCP connection, when
- sending a request, when waiting for a reply, and when receiving a
- reply. If during any of those activities a request fails to make
- forward progress within the alloted time,
- .B httperf
- considers the request to have died, closes the associated connection
- or session and increases the
- .B client\-timo
- error count. The actual timeout value used when waiting for a reply
- is the sum of this timeout and the think\-timeout (see option
- .BR \-\-think\-timeout ).
- By default, the timeout value is infinity.
- .TP
- .BI \-\-uri= S
- Specifies that URI
- .I S
- should be accessed on the server. For some of the workload generators
- (e.g.,
- .BR \-\-wset ),
- this option specifies the prefix for the URIs being accessed.
- .TP
- .BI \-\-use\-timer\-cache
- This feature allows the user to specify whether they want to
- cache timestamps or not. Timestamps are not cached by default, but
- the user can enable caching if higher performance is more important
- than timing accuracy. For small response sizes, disabling timer
- caching reduced the performance of httperf by about
- 10%; for larger response sizes there was little or no effect.
- .TP
- .B \-v
- .TP
- .B \-\-verbose
- Puts
- .B httperf
- into verbose mode. In this mode, additional output such as the
- individual reply rate samples and connection lifetime histogram are
- printed.
- .TP
- .B \-V
- .TP
- .B \-\-version
- Prints the version of
- .BR httperf .
- .TP
- .BI \-\-wlog= B , F
- This option can be used to generate a specific sequence of URI
- accesses. This is useful to replay the accesses recorded in a server
- log file, for example. Parameter
- .I F
- is the name of a file containing the ASCII NUL separated list of URIs
- that should be accessed. If parameter
- .I B
- is set to
- .RB `` y '',
- .B httperf
- will wrap around to the beginning of the file when reaching the end of
- the list (so the list of URIs is accessed repeatedly). With
- .I B
- set to
- .RB `` n '',
- the test will stop no later than when reaching the end of the URI
- list.
- .TP
- .BI \-\-wsess= N1 , N2 , X
- Requests the generation and measurement of sessions instead of
- individual requests. A session consists of a sequence of bursts which
- are spaced out by the user think\-time. Each burst consists of a fixed
- number
- .I L
- of calls to the server
- .RI ( L
- is specified by option
- .BR \-\-burst\-length ).
- The calls in a burst are issued as follows: at first, a single call is
- issued. Once the reply to this first call has been fully received,
- all remaining calls in the burst are issued concurrently. The
- concurrent calls are issued either as pipelined calls on an existing
- persistent connection or as individual calls on separate connections.
- Whether a persistent connection is used depends on whether the server
- responds to the first call with a reply that includes a ``Connection:
- close'' header line. If such a line is present, separate connections
- are used.
-
- The option specifies the following parameters:
- .I N1
- is the total number of sessions to generate,
- .I N2
- is the number of calls per session, and
- .I X
- is the user think\-time (in seconds) that separates consecutive call
- bursts. For example, the options
- .RB `` "\-\-wsess=100,50,10 \-\-burst\-len 5" ''
- would result in 100 sessions with a total of 50 calls each. Since
- each burst has a length of 5 calls, a total of 10 call bursts would be
- generated per session. The user think\-time between call bursts would
- be 10 seconds. Note that user think\-time
- .I X
- denotes the time between receiving the last reply of the previous
- call burst and the sending of the first request of the next burst.
-
- A test involving sessions finishes as soon as the requested number
- .I N1
- of sessions have either failed or completed. A session is considered
- to have failed if any operation in a session takes longer than
- the timeouts specified by options
- .B \-\-timeout
- and
- .BR \-\-think\-timeout .
- In addition, a session also fails if the server returns a reply with a
- status code matching the one specified by option
- .BR \-\-failure\-status .
- .TP
- .BI \-\-wsesslog= N , X , F
- This specifies a session workload generator similar to
- .B \-\-wsess
- (please read that description first). With
- .B \-\-wsesslog
- though, many aspects of user sessions, including the number and
- sequence of URI's, request method, think\-time and burst\-length parameters,
- can be specified in an input file
- .I F.
- Two other parameters are retained from
- .B \-\-wsess,
- namely
- .I N,
- the number of sessions to initiate, and
- .I X,
- the burst\-to\-burst user think time (note that this becomes a default
- time since the input file
- .I F
- can also specify user think time on a per\-burst basis.
- A small example input file can most\-easily show the settable parameters:
- .br
-
- .br
- # Comment lines start with a ``#'' as the first
- .br
- # character. Lines with only whitespace delimit
- .br
- # sessions (multiple blank lines do not generate
- .br
- # ``null'' sessions). All other lines specify a
- .br
- # uri\-sequence (1 uri per line). If the first
- .br
- # character of the line is whitespace (e.g. space
- .br
- # or tab), the uri is considered to be part of a
- .br
- # burst that is sent out after the previous
- .br
- # non\-burst uri.
- .br
-
- .br
- # session 1 definition (this is a comment)
- .br
- /foo.html think=2.0
- .br
- /pict1.gif
- .br
- /pict2.gif
- .br
- /foo2.html method=POST contents='Post data'
- .br
- /pict3.gif
- .br
- /pict4.gif
- .br
-
- .br
- # session 2 definition
- .br
- /foo3.html method=POST contents="Multiline\\ndata"
- .br
- /foo4.html method=HEAD
- .br
-
- .br
- The above description specifies 2 sessions. The first session will
- start with a request for /foo.html. When the /foo.html response comes
- back, a burst of 2 requests will follow (/pict1.gif and /pict2.gif).
- When the last of those responses is received, a two second user think
- time is inserted before the next request of /foo2.html is issued.
- This request is sent as a POST. The posted data can be contained
- between single\- or double\-quotes. Newlines can appear within posted
- data as ``\\n'' or as a ``\\<CR>''. The /foo2.html response is
- followed by a burst request of /pict3.gif and /pict4.gif, which
- concludes this session. The second session is started some time after
- the first, as specified by the
- .B \-\-rate
- or
- .B \-\-period
- options.
- .br
-
- .br
- The second session consists of 2 requests separated by the default user think
- time as specified by the
- .I X
- parameter of the
- .B \-\-wsesslog
- option. If the
- .I N
- parameter of
- .B \-\-wsesslog
- is greater than the number of sessions defined in input file
- .I R F ,
- then the defined sessions are used repeatedly until
- .I N
- sessions have been created (i.e., the defined sessions are used in a
- round\-robin fashion).
- .br
-
- .br
- One should avoid using
- .B \-\-wsesslog
- in conjunction with other
- .B httperf
- options that also control session behavior and workload URI's, namely
- .B \-\-burst\-length,
- .B \-\-wsess,
- .B \-\-wlog,
- and
- .B \-\-wset.
- .TP
- .BI \-\-wset= N , X
- This option can be used to walk through a list of URIs at a given
- rate. Parameter
- .I N
- specifies the number of distinct URIs that should be generated and
- .I X
- specifies the rate at which new URIs are accessed. A rate of
- .B 0.25
- would mean that the same URI would be accessed four times in a row
- before moving on to the next URI. This type of access pattern is
- useful in generating a workload that induces a relatively predictable
- amount of traffic in the disk I/O subsystem of the server (assuming
- .I N
- and the accessed files are big enough to exceed the server's buffer
- cache). The URIs generated are of the form
- .I R prefix / path .html,
- where
- .I prefix
- is the URI prefix specified by option
- .B \-\-uri
- and
- .I path
- is generated as follows: for the
- .I R i \-th
- file in the working set, write down
- .I i
- in decimal, prefixing the number with as many zeroes as necessary
- to get a string that has as many digits as
- .I R N \-1.
- Then insert a slash character between each digit. For example,
- the 103rd file in a working set consisting of 1024 files would
- result in a path of
- .RB `` 0/1/0/3 ''.
- Thus, if the URI\-prefix is
- .BR /wset1024 ,
- then the URI being accessed would be
- .BR /wset1024/0/1/0/3.html .
- In other words, the files on the server need to be organized as a
- 10ary tree.
- .SH "OUTPUT"
- This section describes the statistics output at the end of each test
- run. The basic information shown below is printed independent of the
- selected workload generator.
- .PP
- .RS
- .br
- .B Total:
- connections 30000 requests 29997 replies 29997 test\-duration 299.992 s
- .PP
- .B Connection rate:
- 100.0 conn/s (10.0 ms/conn, <=14 concurrent connections)
- .br
- .B Connection time [ms]:
- min 1.4 avg 3.0 max 163.4 median 1.5 stddev 7.3
- .br
- .B Connection time [ms]:
- connect 0.6
- .br
- .B Connection length [replies/conn]:
- 1.000
- .PP
- .B Request rate:
- 100.0 req/s (10.0 ms/req)
- .br
- .B Request size [B]:
- 75.0
- .PP
- .B Reply rate [replies/s]:
- min 98.8 avg 100.0 max 101.2 stddev 0.3 (60 samples)
- .br
- .B Reply time [ms]:
- response 2.4 transfer 0.0
- .br
- .B Reply size [B]:
- header 242.0 content 1010.0 footer 0.0 (total 1252.0)
- .br
- .B Reply status:
- 1xx=0 2xx=29997 3xx=0 4xx=0 5xx=0
- .PP
- .B CPU time [s]:
- user 94.31 system 205.26 (user 31.4% system 68.4% total 99.9%)
- .br
- .B Net I/O:
- 129.6 KB/s (1.1*10^6 bps)
- .PP
- .B Errors:
- total 3 client\-timo 0 socket\-timo 0 connrefused 3 connreset 0
- .br
- .B Errors:
- fd\-unavail 0 addrunavail 0 ftab\-full 0 other 0
- .br
- .RE
- .PP
- There are six groups of statistics: overall results (``Total''),
- connection related results (``Connection''), results relating to the
- issuing of HTTP requests (``Request''), results relating to the replies
- received from the server (``Reply''), miscellaneous results relating to
- the CPU (``CPU'') and network (``Net I/O'') utilization and, last but not
- least, a summary of errors encountered (``Errors'').
- .TP
- Total Section
- .br
- This section summarizes how many TCP connections were initiated by
- .BR httperf ,
- how many requests it sent out, how many replies it received, and
- what the total test duration was. In the example output
- shown above, 30,000 connections were created, 29,997 requests were
- sent out and 29,997 replies were received. The duration of the
- test was almost exactly 5 minutes (300 seconds).
- .TP
- Connection Section
- .br
- This section conveys information related to TCP connections generated
- by the tool. Specifically, the ``Connection rate'' line shows that new
- connections were initiated at a rate of 100.0 connections per second.
- This rate corresponds to a period of 10.0 milliseconds per
- connection. The last number in this line shows that at most 14
- connections were open at any given time.
-
- The first line labeled ``Connection time'' gives lifetime statistics
- for successful connections. The lifetime of a connection is the time
- between a TCP connection is initiated and the time the connection is
- closed. A connection is considered successful if it had at least one
- call that completed successfully. In the example output, the line
- indicates that the minimum (``min'') connection lifetime was 1.4
- milliseconds, the average (``avg'') lifetime was 3.0 milliseconds, the
- maximum (``max'') was 163.4 milliseconds, the median (``median'')
- lifetime was 1.5 milliseconds, and that the standard deviation of the
- lifetimes was 7.3 milliseconds. The median lifetime is computed based
- on a histogram with one millisecond resolution and a maximum lifetime
- of 100 seconds. Thus, the median is accurate to within half a
- millisecond if at least half of the successful connections have a
- lifetime of no more than 100 seconds.
-
- The next statistic in this section is the average time it took to
- establish a TCP connection. Only successful TCP connection
- establishments are counted. In the example, the second line labeled
- ``Connection time'' shows that, on average, it took 0.6 milliseconds
- to establish a connection.
-
- The final line in this section is labeled ``Connection length.'' It
- gives the average number of replies received on each connection that
- received at least one reply (i.e., connections that failed before
- yielding the first reply are not counted). This number can be bigger
- than 1.0 due to persistent connections.
- .TP
- Request Section
- .br
- The line labeled ``Request rate'' gives the rate at which HTTP requests
- were issued and the period that this rate corresponds to. In the
- example above, the request rate was 100.0 requests per second, which
- corresponds to 10.0 milliseconds per request. As long as no
- persistent connections are employed, the results in this section are
- very similar or identical to results in the connection section.
- However, when persistent connections are used, several calls can be
- performed on a single connection in which case the results would be
- different.
-
- The line labeled ``Request size'' gives the average size of the HTTP
- requests in bytes. In the example above, the average request size was
- 75 bytes.
- .TP
- Reply Section
- .br
- For simple measurements, this section is often the most interesting
- one as the line labeled ``Reply rate'' gives various statistics for
- the reply rate. In the example above, the minimum (``min'') reply
- rate was 98.8 replies per second, the average (``avg'') was 100
- replies per second, and the maximum (``max'') rate was 101.2 replies
- per second. The standard deviation was 0.3 replies per second. The
- number enclosed in parentheses shows that 60 reply rate samples were
- acquired. At present,
- .B httperf
- collects a rate sample once every five seconds. To obtain a
- meaningful standard deviation, it is recommended to run tests long
- enough so at least thirty samples are obtained. This corresponds to a
- test duration of at least 150 seconds.
-
- The line labeled ``Reply Time'' gives information on how long it took
- for the server to respond and how long it took to receive the reply.
- In the example, it took on average 2.4 milliseconds between sending
- the first byte of the request and receiving the first byte of the
- reply. The time to ``transfer'', or read, the reply was too short to
- be measured, so it shows up as zero. The is typical when the entire
- reply fits into a single TCP segment.
-
- The next line, labeled ``Reply size'' contains statistics on the
- average size of the replies\-\-\-all numbers are in reported bytes.
- Specifically, the line lists the average length of reply headers, the
- content, and footers (HTTP/1.1 uses footers to realize the ``chunked''
- transfer encoding). For convenience, the average total number of
- bytes in the replies is also given in parentheses. In the example,
- the average header length (``header'') was 242 bytes, the average
- content length (``content'') was 1010 bytes, and there were no footers
- (``footer'' length is zero). The total reply length of 1252 bytes on
- average.
-
- The final line in this section is a histogram of the major status
- codes received in the replies from the server. The major status code
- is the ``hundreds''\-digit of the full HTTP status code. In the
- example, all 29,997 replies had a major status code of 2. It's a good
- guess that all status codes were ``200 OK'' but the information in the
- histogram is not detailed enough to allow distinguishing status codes
- with the same major code.
- .TP
- Miscellaneous Section
- .br
- This section starts with a summary of the CPU utilization on the
- client machine. In the example, the line labeled ``CPU time'' shows
- that 94.31 seconds were spent executing in user mode (``user''),
- 205.26 seconds were spent executing in system mode (``system'') and
- that this corresponds to 31.4% user mode execution and 68.4% system
- execution. The total utilization was 99.9%, which is expected given
- that
- .B httperf
- is a CPU hog. A total CPU utilization of significantly less than 100%
- is a sign that there were competing processes that interfered with the
- test.
-
- The line labeled ``Net I/O'' gives the average network throughput in
- kilobytes per second (where a kilobyte is 1024 bytes) and in megabits
- per second (where a megabit is 10^6 bits). In the example, an average
- network usage of about 129.6 kilobytes per second was sustained. The
- number in parentheses shows that this corresponds to about 1.1
- megabits per second. This network bandwidth is computed based on the
- number of bytes sent and received on the TCP connections. In other
- words, it does not account for the network headers or TCP
- retransmissions that may have occurred.
- .TP
- Errors Section
- .br
- The last section contains statistics on the errors that were
- encountered during a test. In the example, the two lines labeled
- ``Errors'' show that there were a total of three errors and that all
- three errors were due to the server refusing to accept a connection
- (``connrefused''). A description of each error counter follows:
-
- .B client\-timo:
- The number of times a session, connection, or call failed due
- to a client timeout (as specified by the
- .B \-\-timeout
- and
- .BR \-\-think\-timeout )
- options.
-
- .B socket\-timo:
- The number of times a TCP connection failed with a socket\-level
- timeout (ETIMEDOUT).
-
- .B connrefused:
- The number of times a TCP connection attempt failed with a
- ``connection refused by server'' error (ECONNREFUSED).
-
- .B connreset:
- The number of times a TCP connection failed due to a RESET from the
- server. Typically, a RESET is received when the client attempts to
- send data to the server at a time the server has already closed its
- end of the connection. NT servers also send RESETs when attempting to
- establish a new connection when the listen queue is full.
-
- .B fd\-unavail:
- The number of times the
- .B httperf
- process was out of file descriptors. Whenever this count is non\-zero,
- the test results are meaningless because the client was overloaded
- (see section "CHOOSING TIMEOUT VALUES").
-
- .B addrunavail:
- The number of times the client was out of TCP port numbers
- (EADDRNOTAVAIL). This error should never occur. If it does, the
- results should be discarded.
-
- .B ftab\-full:
- The number of times the system's file descriptor table is full.
- Again, this error should never occur. If it does, the results should
- be discarded.
-
- .B other:
- The number of times some other type of error occurred. Whenever this
- counter is non\-zero, it is necessary to track down the real cause of
- the error. To assist in doing this,
- .B httperf
- prints the error code (errno) of the first unknown errors that occurs
- during a test run.
- .RE
- .PP
- When
- .B \-\-wsess
- or
- .B \-\-wsesslog
- is specified,
- .B httperf
- generates and measures sessions instead of individual calls and
- additional statistics are printed at the end of a test. An example
- output is shown below.
- .PP
- .RS
- .B Session rate [sess/s]:
- min 0.00 avg 0.59 max 2.40 stddev 0.37 (240/450)
- .br
- .B Session:
- avg 6.45 connections/session
- .br
- .B Session lifetime [s]:
- 123.9
- .br
- .B Session failtime [s]:
- 58.5
- .br
- .B Session length histogram:
- 4 7 4 ... 3 3 240
- .RE
- .PP
- The line labeled ``Session rate'' shows the minimum, average, and
- maximum rate at which sessions completed (based on a 5 second sampling
- interval). It also shows the standard deviation of the session
- completion rate. The numbers in parentheses show how many sessions
- succeeded and how many sessions were initiated. In the example above,
- the minimum, average, and maximum session completion rates were 0.00,
- 0.59, and 2.40 sessions per second, respectively. The standard
- deviation was 0.37 sessions per second and 240 out of 450 sessions
- completed successfully (210 failed due to errors such as timeouts).
-
- The next line, labeled ``Session:'' shows the average length of a
- session measured in connections. In the example above, an average of
- 6.45 connections were required to complete a session.
-
- The line labeled ``Session lifetime'' gives the average time it took
- to complete a successful session. In the example above, it took an
- average of 123.9 seconds.
-
- The line labeled ``Session failtime'' gives the average time it took
- before an unsuccessful session failed. In the example above, it took
- on average 58.5 seconds for a session to fail.
-
- Finally, the line labeled ``Session length histogram'' gives a
- histogram of the number of replies received by each session. In the
- example above, 4 sessions ended after receiving no reply at all, 7
- ended after receiving one reply, and so on (the ellipsis indicates
- additional histogram counts that were omitted from this manual for
- space reasons). Note that this histogram does not distinguish between
- successful and failed sessions.
-
- .SH "CHOOSING TIMEOUT VALUES"
- Since the machine that
- .B httperf
- runs on has only a finite set of resource available, it can not
- sustain arbitrarily high HTTP loads. For example, one limiting factor
- is that there are only roughly 60,000 TCP port numbers that can be in
- use at any given time. Since on most UNIX systems it takes one minute
- for a TCP connection to be fully closed (leave the TIME_WAIT state),
- the maximum rate a client can sustain is at most 1,000 requests per
- second.
-
- The actual sustainable rate is often much lower than that because
- before running out of TCP ports, the machine is likely to run out of
- file descriptors (one file descriptor is used up for each open TCP
- connection). By default, HP\-UX 10.20 allows 1,024 open file
- descriptors per process. This means that without extra precautions,
- .B httperf
- could potentially very quickly use up all available file descriptors,
- at which point it could not induce any additional load on the server.
- To avoid this problem,
- .B httperf
- provides option
- .B \-\-timeout
- to set a timeout for all communication with the server. If the server
- does not respond before the timeout expires, the client considers the
- corresponding session, connection, or call to be ``dead,'' closes the
- associated TCP connection, and increases the ``client\-timo'' error
- count. The only exception to this rule is that after sending an
- entire request to the server,
- .B httperf
- allows the server to take some additional time before it starts
- sending the reply. This is to accommodate HTTP requests that take a
- long time to complete on the server. This additional time is called
- the ``server think time'' and can be specified by option
- .BR \-\-think\-timeout .
- By default, this additional think time is zero seconds, so the server
- would always have to respond within the time alloted by option
- .BR \-\-timeout .
-
- Timeouts allow
- .B httperf
- to sustain high offered loads even when the server is overloaded. For
- example, with a timeout of 2 seconds and assuming that 1,000
- file\-descriptors are available, the offered load could be up to 500
- requests per second (in practice, the sustainable load is often
- somewhat smaller than the theoretical value). On the downside,
- timeouts artificially truncate the connection lifetime distribution.
- Thus, it is recommended to pick a timeout value that is as large as
- possible yet small enough to allow sustaining the desired offered
- rate. A timeout as short as one second may be acceptable, but larger
- timeouts (5\-10 seconds) are preferable.
-
- It is important to keep in mind that timeouts do not guarantee that a
- client can sustain a particular offered load\-\-\-there are many other
- potential resource bottlenecks. For example, in some cases the client
- machine may simply run out of CPU time. To ensure that a given test
- really measured the server's capabilities and not the client's, it is
- a good idea to vary the number of machines participating in a test.
- If observed performance remains the same as the number of client
- machines is varied, the test results are likely to be valid.
- .SH "AUTHOR"
- .BR httperf
- was developed by David Mosberger and was heavily influenced by an
- earlier tool written by Tai Jin. Stephane Eranian contributed the
- log\-file based URI generator. Dick Carter contributed the
- .B \-\-wsesslog
- workload generator, the support behind the
- .B \-\-period
- option, and bug fixes. Ted Bullock has taken over maintenance and
- development activities since September 2006.
- .SH "BUGS"
- Probably many. Always be sure to double\-check results and don't fall
- prey to measuring client\-performance instead of server performance!
- .PP
- The user\-interface definitely could be improved. A simple workload
- description language might be more suitable than the dozens of little
- command\-line options the tool has right now.
|