I thought this would be a (probably cross) dupe, but the best I can find is https://unix.stackexchange.com/questions/29402/most-straightforward-way-of-getting-a-raw-unparsed-https-response or https://unix.stackexchange.com/questions/370932/openssl-command-s-client-always-says-400-bad-request . So:
What is an HTTP request? The original version of HTTP used a very simple request format: one line of text consisting of 'GET', a space, and what was originally called the document address (now generalized to resource identifier (URI) locator (URL) name (URN) or path), followed by a line terminator. Because of its severe limitations 0.9 was quickly replaced, and now is often not supported.
HTTP/1.0 in ~1995 expanded this to allow several methods (not just 'GET') and the first line (request line) also includes another space and the protocol version 'HTTP/1.0', and it is followed by zero or more headers each in the form 'name: value' and then (always) one empty line, followed for some requests by a body or 'entity' whose format depends on the headers (and possibly method). In 1.0 none of the headers are mandatory, only the request line and the empty line.
1.0 was quickly superseded by 1.1, which remained predominant, with some tweaks and evolution, until recently, and although formally superseded by 2.0 is still widely used and almost universally supported. 1.1 adds numerous options which can be ignored, but one mandatory request header, namely Host:
to specify, you guessed it, the logical host, allowing multiple websites on one physical server, a practice that is now very common -- enough so that some servers still accept a specification of 1.0 and otherwise follow 1.0 rules but require the Host header to work correctly. (rfc1945 section 19.5 'major differences [from 1.0]' has only one subsection, about requiring the Host: header and allowing absoluteURIs.)
All of these standards specified the line terminator(s) to be the common Internet standard of CRLF (not just LF), but implementations vary on whether they enforce this; see more below.
The simplest way to send an HTTP request over TLS with openssl s_client
is to use its default interactive mode and simply type the HTTP request on the terminal, and wait for the response. If the server chooses to do so, it can renegotiate at this point. An example on my test system, lightly redacted and with comments added:
$ openssl s_client -connect virt2.localdomain:443 -servername virt2.localdomain \
> -CAfile trustvirt2 -cert certs/clix.pem -prexit
<- some servers require SNI and some don't; openssl below 1.1.1 only sends SNI if you specify -servername
<- even though the client cert won't be requested initially I must provide it on the commandline
<- on Unixy systems may need to add -crlf; on Windows don't because it results in CRCRLF which is wrong
CONNECTED(00000003)
depth=0 CN = virt2.localdomain, L = here, C = XX
verify return:1
---
Certificate chain
0 s:/CN=virt2.localdomain/L=here/C=XX
i:/CN=virt2.localdomain/L=here/C=XX
---
Server certificate
-----BEGIN CERTIFICATE-----
[snip]
-----END CERTIFICATE-----
subject=/CN=virt2.localdomain/L=here/C=XX
issuer=/CN=virt2.localdomain/L=here/C=XX
---
No client certificate CA names sent
<- server doesn't do client auth on initial handshake
Server Temp Key: ECDH, prime256v1, 256 bits
---
SSL handshake has read 1550 bytes and written 399 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
[snip]
Verify return code: 0 (ok)
---
GET /secur/good.txt HTTP/1.0
Host: virt2.localdomain
<- I typed those two lines and the empty line
depth=0 CN = virt2.localdomain, L = here, C = XX
verify return:1
<- this output from verify callback shows there was a renegotiation
<- but it is NOT dumped in detail like the initial handshake
read R BLOCK
HTTP/1.1 200 OK
Date: Tue, 05 May 2020 16:27:40 GMT
Server: Apache/2.2.15 (CentOS)
Last-Modified: Mon, 04 May 2020 07:21:53 GMT
ETag: "8066f-7-5a4cd62a20c85"
Accept-Ranges: bytes
Content-Length: 7
Connection: close
Content-Type: text/plain; charset=UTF-8
secret
<- that was the response header and body sent by the server
closed
<- that was output by s_client because my server closed the connection
<- some 1.1 servers might not close immediately here, but will usually timeout fairly soon
<- now s_client -prexit prints the _final_ session data, including the renegotiation:
---
Certificate chain
0 s:/CN=virt2.localdomain/L=here/C=XX
i:/CN=virt2.localdomain/L=here/C=XX
---
Server certificate
-----BEGIN CERTIFICATE-----
[snip]
-----END CERTIFICATE-----
subject=/CN=virt2.localdomain/L=here/C=XX
issuer=/CN=virt2.localdomain/L=here/C=XX
---
Acceptable client certificate CA names
/CN=demoCA2.mytestdomain
/CN=demoCA2b.mytestdomain
<- including the CAnames you want
Server Temp Key: ECDH, prime256v1, 256 bits
---
SSL handshake has read 4252 bytes and written 1909 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
[snip]
Verify return code: 0 (ok)
---
$
There are 2-3 disadvantages to this method:
you must type the request exactly correct, with no mistakes. s_client
sends each line as you enter it, and once sent it cannot be corrected.
depending on the server, you may have to type quickly. Due to widespread denial-of-service attacks, many webservers today only allow a limited time, often 30 seconds or less, to receive the request. This makes the previous bullet even harder.
on Unixy systems, terminal input lines end with only LF, not CRLF. If the server is strict, either you must manually enter the CR, typically as control+V control+M although this can vary depending on your tty driver settings, or (much easier!) add -crlf
to s_client
and it adds CR for you.
The less simple way is to supply the request data automatically: from another command (piped), or a file or heredoc. This avoids the problems of making a manual typing mistake, or being too slow. On Unixy systems where scripts and most commands are LF-only, and files may be, you again have the issue of either adding CR explicitly or using -crlf
. But now after sending the request, you must make s_client
keep the connection open long enough to process the renegotiation -- and maybe, but not necessarily, the server HTTP-level response; this can be done by waiting for terminal input (which you don't immediately provide), by an explicit sleep (at least on Unixy systems), or by adding -ign_eof
to s_client
. Combining these options, some possibilities:
(printf '%s\r\n' 'GET /resource HTTP/1.1' 'Host: myhost' ''; cat) | openssl s_client ...
(echo GET /resource HTTP/1.1; echo Host: myhost; echo; cat) | sed 's/$/\r/' | openssl s_client ...
(echo GET /resource HTTP/1.1; echo Host: myhost; echo; cat) | openssl s_client ... -crlf
(printf '%s\r\n' 'GET /resource HTTP/1.1' 'Host: myhost' ''; sleep 5) | openssl s_client ...
(echo GET /resource HTTP/1.1; echo Host: myhost; echo; sleep 5) | sed 's/$/\r/' | openssl s_client ...
(echo GET /resource HTTP/1.1; echo Host: myhost; echo; sleep 5) | openssl s_client ... -crlf
printf '%s\r\n' 'GET /resource HTTP/1.1' 'Host: myhost' '' | openssl s_client ... -ign_eof
(echo GET /resource HTTP/1.1; echo Host: myhost; echo) | openssl s_client ... -crlf -ign_eof
openssl s_client ... -crlf <<END -ign_eof
GET /resource HTTP/1.1
Host: myhost
END
printf '%s\r\n' 'GET /resource HTTP/1.1' 'Host: myhost' '' >>filewithcr
(echo GET /resource HTTP/1.1; echo Host: myhost; echo) >>filenotcr
cat filewithcr - | openssl s_client ...
(cat filewithcr; sleep 5) | openssl s_client ...
(sed 's/$/\r/' filenotcr; sleep 5) | openssl s_client ...
<filewithcr openssl s_client ... -ign_eof
<filenotcr openssl s_client ... -crlf -ign_eof