Re: Hello from Mozilla

Ian Hickson wrote:
> On Wed, 5 Aug 2009, Amos Jeffries wrote:
>> On Wed, 5 Aug 2009 00:57:18 +0000 (UTC), Ian Hickson <ian_at_hixie.ch> wrote:
>>> On Thu, 30 Jul 2009, Robert Collins wrote:
>>>> On Wed, 2009-07-29 at 23:48 +0000, Ian Hickson wrote:
>>>>> Surely once the HTTP server has decided that it can Upgrade, it
>>>>> doesn't actually need to worry about sending back something
>>>>> HTTP-valid at all, since we can just say the entire connection was
>>>>> always using the WebSocket protocol, and was never HTTP.
>>>> 10.1.2 101 Switching Protocols
>>>>
>>>> The server understands and is willing to comply with the client's
>>>> request, via the Upgrade message header field (section 14.42), for a
>>>> change in the application protocol being used on this connection. The
>>>> server will switch protocols to those defined by the response's
>>>> Upgrade header field immediately after the empty line which
>>>> terminates the 101 response.
>>>>
>>>> So the server has to send a response before switching.
>>> The above text has no normative conformance criteria ("the server
>>> will" is an informative statement that is unsupported as far as I can
>>> tell).
>> Point to note: if webserver fails to switch the WebSockets handshake to
>> verify said switch actually occurred will fail. Detecting such bad servers
>> and acting properly.
>> If web server successfully switched then WebSockets client needs to handle
>> the HTTP response correctly and begin to do a WebSockets native handshake
>> and verify the link is safe _after_ the 101 message is received.
>
> I don't understand what you are trying to say here.
>
>
>>> In any case, the WebSocket protocol handshake does do a complete HTTP
>>> Upgrade including the empty line, it just requires that the handshake
>>> be done in a particular way. It's "profiling" the HTTP spec.
>> No, our whole point repeated over and over in different ways is that the
>> specd description for the upgrade request is _incomplete_. And does not
>> account for an enormous number of byte-level variations. All of which
>> can be safely discarded without affecting security.
>>
>> Being sensitive to whether the server replies "101 Blah" versus "101
>> blah" absolutely cripples WebSockets. We want to help you fix this
>> problem.
>
> I still do not understand why anything gets crippled. Maybe you could show
> an example of how you expect this problem to occur?
>

Your protocol definition used byte-level. At the byte-level 'b' (0x97
IIRC) does not equal 'B' (0x65 IIRC). Thus the response is a different
byte pattern and a failed WebSocket connection.

One very real example of this would be the web server or an fully
WebSocket capable intermediary sending back bytes

Your spec section 3.1 sub 12 says:
"
  12. Read the first 85 bytes from the server. If the connection
         closes before 85 bytes are received, or if the first 85 bytes
         aren't exactly equal to the following bytes, then fail the Web
"
   [ note the words _exactly equal_ ]
"
         Socket connection and abort these steps.

            48 54 54 50 2f 31 2e 31 20 31 30 31 20 57 65 62
            20 53 6f 63 6b 65 74 20 50 72 6f 74 6f 63 6f 6c
            20 48 61 6e 64 73 68 61 6b 65 0d 0a 55 70 67 72
            61 64 65 3a 20 57 65 62 53 6f 63 6b 65 74 0d 0a
            43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 55 70 67 72
            61 64 65 0d 0a
"

example #1 suppose there was an intermediary translating
websockets-over-http to websockets-port-81 which used HTTP to format
said headers of confirmation.
In all other ways it is fully WebSockets compliant. But sends byte 18 as
73 (s) instead of 53 (S).
Boom! The entire application is not WebSockets compliant and will fail
every single transaction that goes through it.

example #2 is where the traffic is processed by an HTTP-only
intermediary which sees the 'Upgrade:' header and flags the connection
for transparent pass-thru (This by the way is the desirable method of
making Squid support WebSockets).

Being a good HTTP relay it accepts these bytes:
   HTTP/1.1 101 Web Socket Protocol Handshake
   Upgrade: WebSocket
   Connection: Upgrade

It violates HTTP by omitting the Via and other headers your spec omits
to handle. And passes these on:
   HTTP/1.1 101 Web Socket Protocol Handshake
   Connection: Upgrade
   Upgrade: WebSocket

then moves to tunnel mode for you.

Bang. Another whole network isolated from WebSockets.

>
>>>> This is because if it doesn't, how can it say 'I won't upgrade'.
>>> It can just not upgrade. Returning anything but the correct handshake
>>> will be treated as a failed connection by the WebSocket client.
>> Correct.
>
> So why would it need to say "I won't upgrade"?

To inform MITM that the upgrade is not going to happen and the links
they have open maybe used for other HTTP things without wasting network
resources tearing them down and rebuilding.

>
>>> On Thu, 30 Jul 2009, Robert Collins wrote:
>>>>> Suppose we had no handshake at all, and that there was no data
>>>>> framing, so that as soon as we connected to a port, we could send
>>>>> arbitrary data down.
>>>>>
>>>>> A Web page, say evil.example.net, could open a Web Socket
>>>>> connection to http://www.corp.example.com/, send it a GET request
>>>>> for /secret-plans, and then forward the contents of the file to a
>>>>> remote host. If they could then trick someone on example.com's
>>>>> intranet to look at this file, and assuming www.corp.example.com
>>>>> did nothing more than rely on connectivitity for authentication
>>>>> (pretty common in small intranets), then evil.example.net could
>>>>> steal the company's secret plans.
>>>> Can't the web page just send an ajax request to corp.example.com
>>>> anyway?
>>> It can't read the response, no.
>> Can you explain this please?
>> AFAIK, Sending a request then closing the connection immediately is the
>> only way said response might be unreadable. I don't understand what you
>> mean.
>
> A script on a Web page can cause a GET request to be sent to an arbitrary
> URL, but it has no way to obtain the contents of the result of that
> request unless that server opts in (using CORS) to allowing the script to
> see the contents of the response.

So in other words:

   the remote web server being queried has to accept the TCP connection,
perform the correct HTTP-level handshakes, then correct CORS handshakes
in order for you to use those links?

as compared to:

  the remote web server being queried has to accept the TCP connection,
perform the correct HTTP-level handshakes, then correct WebSockets
handshakes in order for you to use those links?

(note the single word difference).

>
>>>> Dumping the ability to look like HTTP altogether is your best bet
>>>> IMO: even on port 443. For port 443, someone running a websocket
>>>> server should just not run an HTTPS server there. Or define port 815
>>>> as websocket/s.
>>> Port 815 is the preferred port, but in certain situations, only ports
>>> 80 and 443 are allowed, and port 80 is mitm'ed, so that only leaves
>>> 443.
>>>
>>> You don't always have the option of using a different host, that's why
>>> we need to share the port sometimes. (There have also been statements
>>> to the effect that just sending a new protocol over ports 80 or 443
>>> without using the Upgrade mechanism would be a violation of the
>>> semantics of those ports.)
>> Understood. However strange protocols (torrent is one example) on port 80
>> can be detected and dealt with easily.
>> Something that appears to be perfect HTTP cannot be detected as non-HTTP.
>> When HTTP operations are done to the traffic it will break badly and in
>> nasty invisible ways.
>
> Could you explain how a WebSocket connection can break in an invisible
> way? An example would be very helpful here.

see every single example so far posted. Particularly example #2 this post.

>
>>>> If its not, it should bail. That seems reasonable :). We can
>>>> definitely (and have been :)) advising on how to do that with HTTP,
>>>> but it means *having a decent HTTP stack*. The response to a HTTP
>>>> request is always an HTTP response. Upgrade: changes the way the wire
>>>> behaves after that.
>>> It seems that just having a WebSocket "stack" is adequate also.
>> When all layers of the stack confirm to the appropriate protocols, yes
>> agreed.
>> Point of this discussion is that the WebSockets layer _when sitting on top
>> of HTTP_ needs to be using HTTP layer correctly.
>
> WebSockets, even when initiating the connection by talking to an HTTP
> server and then switching to WebSockets, isn't layered on HTTP. It's just
> doing the bare minimum required to allow the HTTP server to understand the
> request and get out of the way. Once the handshake is complete, there is
> no HTTP anywhere on the stack at all.

Its not doing the bare minimum.
The bare minimum would be to accept the valid HTTP transforms which the
Internet _will_ perform on the handshake. Discard those useless
transforms and validate the handshake status line.

>
>>>>> Yes, but the client is a WebSocket client, not an HTTP client, so
>>>>> why would it send anything but the WebSocket handshake?
>>>> Because its asking an HTTP server to upgrade.
>>> How can it know that ahead of time?
>> When using port 80 is must assume to be doing so.
>> a) because it _is_
>> b) because any WebSockets server on port 80 is expecting that HTTP upgrade
>> request of yours to come in.
>>
>> Therefore, doing proper HTTP Upgrade and discarding the junk headers is
>> no loss, remains secure, and a net gain when MITM are present doing
>> perfectly normal HTTP things.
>
> I disagree that it remains secure, but leaving that aside, could you
> explain how it is a net gain when MITM are present? Under what conditions
> would a MITM allow a WebSocket Upgrade to actually succeed end-to-end,
> even if we do everything according to HTTP's rules instead of the
> simplified WebSocket rules?

see example #2 outlined above.

>
>>>> That makes it an HTTP client and a WebSocket client... as in fact you
>>>> note below ..
>>> It's both. It's not an HTTP client in that it doesn't do anything but
>>> what the WebSocket spec says it should do. It's an HTTP client in that
>>> it speaks something that can be interpreted as HTTP.
>> Your spec says otherwise. Simply by defining the byte-level content of
>> the headers the server and client are now speaking non-HTTP. Were they
>> spec'd as speaking HTTP this entire argument would not be ocurring.
>
> They're not speaking HTTP, they're speaking something that can be
> interpreted as HTTP. I think the distinction is quite important.

Correct the distinction _is_ important. Add to it the port they are
speaking said 'something' through.

It becomes a critical point of failure if said 'something' should be
sent over port 80, port 3128.

It becomes an annoyance to network admin, but not critical if said
'something' is sent over port 81, port 88, port 8080, and port 8000.
Also a range of other ports with even lesser impact.

>>>>>> Ian, are you absolutely certain that everywhere you use "the
>>>>>> internet", there is no "man in the middle" between you and the
>>>>>> server you're speaking to?
>>>>> In the case of TLS connections, yes.
>>>> In corporate networking, TLS MITM is a 'feature': company signed
>>>> certificates are used to sign the TLS connection to the corporate
>>>> firewall, and the firewall validate the SSL connection to the outside
>>>> world. I haven't personally used this, so can't really say much more
>>>> about it. Squid's ssl-bump feature can be used for this, but I
>>>> believe browser config is needed (again on a corporate basis) to tell
>>>> it that this is expected.
>>> I can't think of any way of getting a WebSocket connection through
>>> such a situation, so there's not much point doing anything to handle
>>> this case. If you can think of a way of handling it that _would_ allow
>>> WebSocket through, let me know, as then it would make sense to support
>>> it.
>> Don't confuse TLS MITM with regular MITM. Regular MITM are still very
>> common and can be tunneled over without trouble just by accepting
>> regular HTTP changes during the upgrade request and reply headers.
>
> Could you show me a proof of concept of this? I've not been able to get
> any sort of two-way connection to a remote host when there's a MITM proxy
> in the middle of the connection, regardless of what packets I send. Any
> help you could provide here would be much appreciated.
>

It's not the sending that matters so much. It's the receiving.

Write your code as I spec'd out for you earlier.
  * Send what you want.
  * Receive whatever HTTP arrives and discard/skip/ignore all the
useless headers that come back before the first pair of 'CRLFCRLF' (two
sequential CRLF).
  * The bytes that follow after the 101 reply (for Upgrade:) or 200 (for
CONNECT) are your two-way TCP link.

What MITM are you testing with?

Amos

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE18
   Current Beta Squid 3.1.0.13