Phase should only be accessed under in.Mutex. Handshake and all Read
operations obtain that lock. However, many functions checking for
handshakeRunning only obtain handshakeMutex: reintroduce
handshakeCompleted for them. ConnectionState and Close check for
handshakeConfirmed, introduce an atomic flag for them.
There is no reason a server can't just send a CloseNotify in its first
flight, and then close the connection without reading the 0-RTT data.
Also, it's not expected of Close to block on reading, and interlocking
with a Read can cause a deadlock.
Fixes NCC-2016-001
Opening the 1.3 dances with the record layer because it has been the
most stable through the drafts, has the least dependencies, and has been
tricky in my experience.
Note that the record layer version check is entirely removed according
to https://tools.ietf.org/html/draft-ietf-tls-tls13-18#appendix-C.2.
A test that happened to hit that check (but was not made to test for it)
has changed to the next error in the stack.
There are no 1.3 tests at the moment, and I suspect they will all have to
wait for the patch cycle to reach interoperability.
Using > / <= VersionTLS13 for all conditionals to transparently support
draft versions and hypotetical future versions.
See https://tools.ietf.org/html/draft-ietf-tls-tls13-18#section-5.
Updates #9671
Change-Id: I97f0a59439728f194a1c50b48cff041469a0f00b
There's no need to hold the handshake lock across this call and it can
lead to deadlocks if the net.Conn calls back into the tls.Conn.
Fixes#18426.
Change-Id: Ib1b2813cce385949d970f8ad2e52cfbd1390e624
Reviewed-on: https://go-review.googlesource.com/36561
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The tree is inconsistent about single l vs double l in those
words in documentation, test messages, and one error value text.
$ git grep -E '[Mm]arshall(|s|er|ers|ed|ing)' | wc -l
42
$ git grep -E '[Mm]arshal(|s|er|ers|ed|ing)' | wc -l
1694
Make it consistently a single l, per earlier decisions. This means
contributors won't be confused by misleading precedence, and it helps
consistency.
Change the spelling in one error value text in newRawAttributes of
crypto/x509 package to be consistent.
This change was generated with:
perl -i -npe 's,([Mm]arshal)l(|s|er|ers|ed|ing),$1$2,' $(git grep -l -E '[Mm]arshall' | grep -v AUTHORS | grep -v CONTRIBUTORS)
Updates #12431.
Follows https://golang.org/cl/14150.
Change-Id: I85d28a2d7692862ccb02d6a09f5d18538b6049a2
Reviewed-on: https://go-review.googlesource.com/33017
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The CloseWrite method sends a close_notify alert record to the other
side of the connection. This record indicates that the sender has
finished sending on the connection. Unlike the Close method, the sender
may still read from the connection until it recieves a close_notify
record (or the underlying connection is closed). This is analogous to a
TCP half-close.
This is a rework of CL 25159 with fixes for the unstable test.
Updates #8579
Change-Id: I47608d2f82a88baff07a90fd64c280ed16a60d5e
Reviewed-on: https://go-review.googlesource.com/31318
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This reverts commit c6185aa63217c84a1a73c578c155e7d4dec6cec8. That
commit seems to be causing flaky failures on the builders. See
discussion on the original thread: https://golang.org/cl/25159.
Change-Id: I26e72d962d4efdcee28a0bc61a53f246b046df77
Reviewed-on: https://go-review.googlesource.com/31316
Run-TryBot: Adam Langley <agl@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This change adds support for the ChaCha20-Poly1305 AEAD to crypto/tls,
as specified in https://tools.ietf.org/html/rfc7905.
Fixes#15499.
Change-Id: Iaa689be90e03f208c40b574eca399e56f3c7ecf1
Reviewed-on: https://go-review.googlesource.com/30957
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The CloseWrite method sends a close_notify alert record to the other
side of the connection. This record indicates that the sender has
finished sending on the connection. Unlike the Close method, the sender
may still read from the connection until it recieves a close_notify
record (or the underlying connection is closed). This is analogous to a
TCP half-close.
Updates #8579
Change-Id: I9c6bc193efcb25cc187f7735ee07170afa7fdde3
Reviewed-on: https://go-review.googlesource.com/25159
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The aim is to make the decrypt() timing profile constant, irrespective of
the CBC padding length or correctness. The old algorithm, on valid padding,
would only MAC bytes up to the padding length threshold, making CBC
ciphersuites vulnerable to plaintext recovery attacks as presented in the
"Lucky Thirteen" paper.
The new algorithm Write()s to the MAC all supposed payload, performs a
constant time Sum()---which required implementing a constant time Sum() in
crypto/sha1, see the "Lucky Microseconds" paper---and then Write()s the rest
of the data. This is performed whether the padding is good or not.
This should have no explicit secret-dependent timings, but it does NOT
attempt to normalize memory accesses to prevent cache timing leaks.
Updates #13385
Change-Id: I15d91dc3cc6eefc1d44f317f72ff8feb0a9888f7
Reviewed-on: https://go-review.googlesource.com/18130
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
readRecord was not returning early if c.in.decrypt failed and ran
through the rest of the function. It does set c.in.err, so the various
checks in the callers do ultimately notice before acting on the result,
but we should avoid running the rest of the function at all.
Also rename 'err' to 'alertValue' since it isn't actually an error.
Change-Id: I6660924716a85af704bd3fe81521b34766238695
Reviewed-on: https://go-review.googlesource.com/24709
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
After renegotiation support was added (af125a5193c) it's possible for a
Write to block on a Read when racing to complete the handshake:
1. The Write determines that a handshake is needed and tries to
take the neccesary locks in the correct order.
2. The Read also determines that a handshake is needed and wins
the race to take the locks.
3. The Read goroutine completes the handshake and wins a race
to unlock and relock c.in, which it'll hold when waiting for
more network data.
If the application-level protocol requires the Write to complete before
data can be read then the system as a whole will deadlock.
Unfortunately it doesn't appear possible to reverse the locking order of
c.in and handshakeMutex because we might read a renegotiation request at
any point and need to be able to do a handshake without unlocking.
So this change adds a sync.Cond that indicates that a goroutine has
committed to doing a handshake. Other interested goroutines can wait on
that Cond when needed.
The test for this isn't great. I was able to reproduce the deadlock with
it only when building with -race. (Because -race happened to alter the
timing just enough.)
Fixes#17101.
Change-Id: I4e8757f7b82a84e46c9963a977d089f0fb675495
Reviewed-on: https://go-review.googlesource.com/29164
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Since 2a8c81ff handshake messages are not written directly to wire but
buffered. If an error happens at the wrong time the alert will be
written to the buffer but never flushed, causing an EOF on the client
instead of a more descriptive alert.
Thanks to Brendan McMillion for reporting this.
Fixes#17037
Change-Id: Ie093648aa3f754f4bc61c2e98c79962005dd6aa2
Reviewed-on: https://go-review.googlesource.com/28818
Reviewed-by: Adam Langley <agl@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Moves the state.ServerName assignment to outside the if
statement that checks for handshakeComplete.
Fixes#15571
Change-Id: I6c4131ddb16389aed1c410a975f9aa3b52816965
Reviewed-on: https://go-review.googlesource.com/22862
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
This change causes TLS handshake messages to be buffered and written in
a single Write to the underlying net.Conn.
There are two reasons to want to do this:
Firstly, it's slightly preferable to do this in order to save sending
several, small packets over the network where a single one will do.
Secondly, since 37c28759ca46cf381a466e32168a793165d9c9e9 errors from
Write have been returned from a handshake. This means that, if a peer
closes the connection during a handshake, a “broken pipe” error may
result from tls.Conn.Handshake(). This can mask any, more detailed,
fatal alerts that the peer may have sent because a read will never
happen.
Buffering handshake messages means that the peer will not receive, and
possibly reject, any of a flow while it's still being written.
Fixes#15709
Change-Id: I38dcff1abecc06e52b2de647ea98713ce0fb9a21
Reviewed-on: https://go-review.googlesource.com/23609
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The current code, introduced after Go 1.6 to improve latency on
low-bandwidth connections, sends 1 kB packets until 1 MB has been sent,
and then sends 16 kB packets (the maximum record size).
Unfortunately this decreases throughput for 1-16 MB responses by 20% or so.
Following discussion on #15713, change cutoff to 128 kB sent
and also grow the size allowed for successive packets:
1 kB, 2 kB, 3 kB, ..., 15 kB, 16 kB.
This fixes the throughput problems: the overhead is now closer to 2%.
I hope this still helps with latency but I don't have a great way to test it.
At the least, it's not worse than Go 1.6.
Comparing MaxPacket vs DynamicPacket benchmarks:
name maxpkt time/op dyn. time/op delta
Throughput/1MB-8 5.07ms ± 7% 5.21ms ± 7% +2.73% (p=0.023 n=16+16)
Throughput/2MB-8 15.7ms ±201% 8.4ms ± 5% ~ (p=0.604 n=20+16)
Throughput/4MB-8 14.3ms ± 1% 14.5ms ± 1% +1.53% (p=0.000 n=16+16)
Throughput/8MB-8 26.6ms ± 1% 26.8ms ± 1% +0.47% (p=0.003 n=19+18)
Throughput/16MB-8 51.0ms ± 1% 51.3ms ± 1% +0.47% (p=0.000 n=20+20)
Throughput/32MB-8 100ms ± 1% 100ms ± 1% +0.24% (p=0.033 n=20+20)
Throughput/64MB-8 197ms ± 0% 198ms ± 0% +0.56% (p=0.000 n=18+7)
The small MB runs are bimodal in both cases, probably GC pauses.
But there's clearly no general slowdown anymore.
Fixes#15713.
Change-Id: I5fc44680ba71812d24baac142bceee0e23f2e382
Reviewed-on: https://go-review.googlesource.com/23487
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change adds Config.Renegotiation which controls whether a TLS
client will accept renegotiation requests from a server. This is used,
for example, by some web servers that wish to “add” a client certificate
to an HTTPS connection.
This is disabled by default because it significantly complicates the
state machine.
Originally, handshakeMutex was taken before locking either Conn.in or
Conn.out. However, if renegotiation is permitted then a handshake may
be triggered during a Read() call. If Conn.in were unlocked before
taking handshakeMutex then a concurrent Read() call could see an
intermediate state and trigger an error. Thus handshakeMutex is now
locked after Conn.in and the handshake functions assume that Conn.in is
locked for the duration of the handshake.
Additionally, handshakeMutex used to protect Conn.out also. With the
possibility of renegotiation that's no longer viable and so
writeRecordLocked has been split off.
Fixes#5742.
Change-Id: I935914db1f185d507ff39bba8274c148d756a1c8
Reviewed-on: https://go-review.googlesource.com/22475
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Error strings in this package were all over the place: some were
prefixed with “tls:”, some with “crypto/tls:” and some didn't have a
prefix.
This change makes everything use the prefix “tls:”.
Change-Id: Ie8b073c897764b691140412ecd6613da8c4e33a2
Reviewed-on: https://go-review.googlesource.com/21893
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
This change removes a lot of dead code. Some of the code has never been
used, not even when it was first commited. The rest shouldn't have
survived refactors.
This change doesn't remove unused routines helpful for debugging, nor
does it remove code that's used in commented out blocks of code that are
only unused temporarily. Furthermore, unused constants weren't removed
when they were part of a set of constants from specifications.
One noteworthy omission from this CL are about 1000 lines of unused code
in cmd/fix, 700 lines of which are the typechecker, which hasn't been
used ever since the pre-Go 1 fixes have been removed. I wasn't sure if
this code should stick around for future uses of cmd/fix or be culled as
well.
Change-Id: Ib714bc7e487edc11ad23ba1c3222d1fd02e4a549
Reviewed-on: https://go-review.googlesource.com/20926
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently, if a client of crypto/tls (e.g., net/http, http2) calls
tls.Conn.Write with a 33KB buffer, that ends up writing three TLS
records: 16KB, 16KB, and 1KB. Slow clients (such as 2G phones) must
download the first 16KB record before they can decrypt the first byte.
To improve latency, it's better to send smaller TLS records. However,
sending smaller records adds overhead (more overhead bytes and more
crypto calls), which slightly hurts throughput.
A simple heuristic, implemented in this change, is to send small
records for new connections, then boost to large records after the
first 1MB has been written on the connection.
Fixes#14376
Change-Id: Ice0f6279325be6775aa55351809f88e07dd700cd
Reviewed-on: https://go-review.googlesource.com/19591
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Tom Bergan <tombergan@google.com>
Reviewed-by: Adam Langley <agl@golang.org>
This change improves the error message when encountering a TLS handshake
message that is larger than our limit (64KB). Previously the error was
just “local error: internal error”.
Updates #13401.
Change-Id: I86127112045ae33e51079e3bc047dd7386ddc71a
Reviewed-on: https://go-review.googlesource.com/20547
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This promotes a connection hang during TLS handshake to a proper error.
This doesn't fully address #14539 because the error reported in that
case is a write-on-socket-not-connected error, which implies that an
earlier error during connection setup is not being checked, but it is
an improvement over the current behaviour.
Updates #14539.
Change-Id: I0571a752d32d5303db48149ab448226868b19495
Reviewed-on: https://go-review.googlesource.com/19990
Reviewed-by: Adam Langley <agl@golang.org>
The tree's pretty inconsistent about single space vs double space
after a period in documentation. Make it consistently a single space,
per earlier decisions. This means contributors won't be confused by
misleading precedence.
This CL doesn't use go/doc to parse. It only addresses // comments.
It was generated with:
$ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])')
$ go test go/doc -update
Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7
Reviewed-on: https://go-review.googlesource.com/20022
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Dave Day <djd@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Conn.Close sends an encrypted "close notify" to signal secure EOF.
But writing that involves acquiring mutexes (handshake mutex + the
c.out mutex) and writing to the network. But if the reason we're
calling Conn.Close is because the network is already being
problematic, then Close might block, waiting for one of those mutexes.
Instead of blocking, and instead of introducing new API (at least for
now), distinguish between a normal Close (one that sends a secure EOF)
and a resource-releasing destructor-style Close based on whether there
are existing Write calls in-flight.
Because io.Writer and io.Closer aren't defined with respect to
concurrent usage, a Close with active Writes is already undefined, and
should only be used during teardown after failures (e.g. deadlines or
cancelations by HTTP users). A normal user will do a Write then
serially do a Close, and things are unchanged for that case.
This should fix the leaked goroutines and hung net/http.Transport
requests when there are network errors while making TLS requests.
Change-Id: If3f8c69d6fdcebf8c70227f41ad042ccc3f20ac9
Reviewed-on: https://go-review.googlesource.com/18572
Reviewed-by: Adam Langley <agl@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The user can inspect the record data to detect that the other side is
not using the TLS protocol.
This will be used by the net/http client (in a follow-on CL) to detect
when an HTTPS client is speaking to an HTTP server.
Updates #11111.
Change-Id: I872f78717aa8e8e98cebd8075436209a52039a73
Reviewed-on: https://go-review.googlesource.com/16078
Reviewed-by: Adam Langley <agl@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The one in misc/makerelease/makerelease.go is particularly bad and
probably warrants rotating our keys.
I didn't update old weekly notes, and reverted some changes involving
test code for now, since we're late in the Go 1.5 freeze. Otherwise,
the rest are all auto-generated changes, and all manually reviewed.
Change-Id: Ia2753576ab5d64826a167d259f48a2f50508792d
Reviewed-on: https://go-review.googlesource.com/12048
Reviewed-by: Rob Pike <r@golang.org>
The OCSP response is currently only exposed via a method on Conn,
which makes it inaccessible when using wrappers like net/http. The
ConnectionState structure is typically available even when using
wrappers and contains many of the other handshake details, so this
change exposes the stapled OCSP response in that structure.
Change-Id: If8dab49292566912c615d816321b4353e711f71f
Reviewed-on: https://go-review.googlesource.com/9361
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
This change adds support for serving and receiving Signed Certificate
Timestamps as described in RFC 6962.
The server is now capable of serving SCTs listed in the Certificate
structure. The client now asks for SCTs and, if any are received,
they are exposed in the ConnectionState structure.
Fixes#10201
Change-Id: Ib3adae98cb4f173bc85cec04d2bdd3aa0fec70bb
Reviewed-on: https://go-review.googlesource.com/8988
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
Reviewed-by: Jonathan Rudenberg <jonathan@titanous.com>
Some servers which misunderstood the point of the CertificateRequest
message send huge reply records. These records are large enough that
they were considered “insane” by the TLS code and rejected.
This change removes the sanity test for record lengths. Although the
maxCiphertext test still remains, just above, which (roughly) enforces
the 16KB protocol limit on record sizes:
https://tools.ietf.org/html/rfc5246#section-6.2.1Fixes#8928.
Change-Id: Idf89a2561b1947325b7ddc2613dc2da638d7d1c9
Reviewed-on: https://go-review.googlesource.com/5690
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Update #3514
An io.Reader is permitted to return either (n, nil)
or (n, io.EOF) on EOF or other error.
The tls package previously always returned (n, nil) for a read
of size n if n bytes were available, not surfacing errors at
the same time.
Amazon's HTTPS frontends like to hang up on clients without
sending the appropriate HTTP headers. (In their defense,
they're allowed to hang up any time, but generally a server
hangs up after a bit of inactivity, not immediately.) In any
case, the Go HTTP client tries to re-use connections by
looking at whether the response headers say to keep the
connection open, and because the connection looks okay, under
heavy load it's possible we'll reuse it immediately, writing
the next request, just as the Transport's always-reading
goroutine returns from tls.Conn.Read and sees (0, io.EOF).
But because Amazon does send an AlertCloseNotify record before
it hangs up on us, and the tls package does its own internal
buffering (up to 1024 bytes) of pending data, we have the
AlertCloseNotify in an unread buffer when our Conn.Read (to
the HTTP Transport code) reads its final bit of data in the
HTTP response body.
This change makes that final Read return (n, io.EOF) when
an AlertCloseNotify record is buffered right after, if we'd
otherwise return (n, nil).
A dependent change in the HTTP code then notes whether a
client connection has seen an io.EOF and uses that as an
additional signal to not reuse a HTTPS connection. With both
changes, the majority of Amazon request failures go
away. Without either one, 10-20 goroutines hitting the S3 API
leads to such an error rate that empirically up to 5 retries
are needed to complete an API call.
LGTM=agl, rsc
R=agl, rsc
CC=golang-codereviews
https://golang.org/cl/76400046
Currently a write error will cause future reads to return that same error.
However, there may have been extra information from a peer pending on
the read direction that is now unavailable.
This change splits the single connErr into errors for the read, write and
handshake. (Splitting off the handshake error is needed because both read
and write paths check the handshake error.)
Fixes#7414.
LGTM=bradfitz, r
R=golang-codereviews, r, bradfitz
CC=golang-codereviews
https://golang.org/cl/69090044
