mirror/cypress
1
0
Fork 0
mirror of https://github.com/cypress-io/cypress.git synced 2026-05-21 07:38:53 -05:00
Code Issues Packages Projects Releases Wiki Activity

Improved proxy support (#3531)

Browse Source 
* https-proxy: unused file

* server: wrap all https requests that use a proxy

* server: use request lib in ensureUrl if proxy is in use. this makes runs tab work behind a proxy

* electron: pass --proxy-server to app itself, so the embedded github login page works

* cli: first attempt at env vars from windows registry

* cli: api cleanup

* cli: lint

* cli: fix crash on no proxy, add tests

* add desktop-gui watch to terminals.json

* cli: pass along --proxy-source

* electron: pass --proxy-bypass-list too

* server: whitelist proxy* args

* cli: better wording

* desktop-gui: display proxy settings

* extension: force proxy [wip]

* extension: finally, i am victorious over coffeescript

* extension: add -loopback to bypasslist

* extension: revert changes

Revert "extension: force proxy [wip]"

This reverts commit 3ab6ba42a763f25ee65f12eb8b79eb597efc9b11.

* desktop-gui: skip proxysettings if there aren't any

* https-proxy, server: proxy directConnections using https-proxy-agent

* https-agent: pool httpsAgents

* https-proxy: work when they're not on a proxy

* https-proxy: ci - use agent 1.0

* https-proxy: tests

* desktop-gui: hide proxy settings when not using proxy

* https-proxy: pass req through to https-proxy-agent callback

* cli: use get-windows-proxy

* desktop-gui: always show proxy settings

* server: use get-windows-proxy

* electron, server: supply electron proxy config when window launched

* server: fix

* https-proxy: cleanup

* server: clean up ensureUrl

* https-proxy: cleanup

* cli: fix

* cli: fix destructuring

* server: enable ForeverAgent to pool HTTPS/HTTP connections

#3192

* server: updating snapshot

* https-proxy: don't crash, do error if proxy unreachable

* https-proxy:

* get-windows-proxy@1.0.0

* https-proxy: use proxy-from-env to decide on a proxy for a url

* server: fallback to HTTP_PROXY globally if HTTPS_PROXY not set

* server: proxy args test

* cli: add proxy tests

* cli: add test that loadSystemProxySettings is called during download

* cli, server: account for the fact that CI has some proxy vars set

* https-proxy: ""

* cli, https-proxy, server: ""

* desktop-gui: update settings gui

* desktop-gui: cypress tests for proxy settings

* server: strict undefined check

* cli, server: move get-windows-proxy to scope, optionalDeps

* server, cli: use new and improved get-windows-proxy

* cli, server: 1.5.0

* server: re-check for proxy since cli may have failed to load the lib

* server, cli: 1.5.1

* server: NO_PROXY=localhost by default, clean up

* https-proxy: disable Nagle's on proxy sockets

\#3192

* https-proxy: use setNoDelay on upstream, cache https agent

* https-proxy: test basic auth

* https-proxy: add todo: remove this

* server: add custom HTTP(s) Agent implementation w keepalive, tunneling

* server: typescript for agent

* add ts to zunder

* server: more ts

* ts: add missing Agent type declaration

* server: create CombinedAgent

* server: use agent in more places

* ts: more declarations

* server: make script work even if debug port not supplied

* server: begin some testing

* server, ts: agent, tests

* server: test

* server: agent works with websockets now

* server: update snapshot

* server: work out some more bugs with websockets

* server: more websockets

* server: add net_profiler

* https-proxy: fix dangling socket on direct connection

* server: fix potential 'headers have already been sent'

* https-proxy: nab another dangler

* server: update test to expect agent

* https-proxy: fix failing test

* desktop-gui: change on-link

* server: add tests for empty response case

* server: tests

* server: send keep-alive with requests

* server: make net profiler hook on socket.connect

* server: only hook profiler once

* server: update tests, add keep-alive test

* server: only regen headers if needed

* server: move http_overrides into CombinedAgent, make it proxy-proof

for #112

* server: update snapshot

* server: undo

* server: avoid circular dependency

* https-proxy, server: use our Agent instead of https-proxy-agent

* server: add dependency back

* cli: actually use proxy for download

* server, launcher, ts: typescript

* Revert "server, launcher, ts: typescript"

This reverts commit d3f8b8bbb6.

* Revert "Revert "server, launcher, ts: typescript""

This reverts commit 818dfdfd00.

* ts, server: respond to PR

* server, ts: types

* ts: really fix types

* https-proxy, server: export CA from https-proxy

* agent, server, https-proxy: move agent to own package

* agent => networking, move connect into networking

* fix tests

* fix test

* networking: respond to PR changes, add more unit tests

* rename ctx

* networking, ts: add more tests

* server: add ensureUrl tests

* https-proxy: remove https-proxy-agent

* server: use CombinedAgent for API

* server: updates

* add proxy performance tests

* add perf tests to workflow

* circle

* run perf tests with --no-sandbox

* networking, ts: ch-ch-ch-ch-changes

* server, networking: pr changes

* run networking tests in circle

* server: fix performance test

* https-proxy: test that sockets are being closed

* https-proxy: write, not emit

* networking: fix test

* networking: bubble err in connect

* networking: style

* networking: clean p connect error handling

* networking => network

* server: make perf tests really work

* server: really report

* server: use args from browser

* server: use AI to determine max run time

* server: load electron only when needed


Co-authored-by: Brian Mann <brian@cypress.io>
This commit is contained in:
Zach Bloomquist
2019-03-31 23:39:10 -04:00
committed by Brian Mann
parent c4e90956de
commit c1a345dce2
66 changed files with 2317 additions and 477 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/https-proxy/test/helpers/mitm.coffee
-17
View File
@@ -1,17 +0,0 @@
Promise = require("bluebird")
Proxy = require("http-mitm-proxy")
proxy = Proxy()
proxy.onRequest (ctx, cb) ->
cb()
module.exports = {
start: ->
new Promise (resolve) ->
proxy.listen({port: 8081, forceSNI: true}, resolve)
stop: ->
proxy.close()
}
packages/https-proxy/test/helpers/proxy_bak.coffee
-284
View File
@@ -1,284 +0,0 @@
fs = require("fs-extra")
net = require("net")
url = require("url")
path = require("path")
http = require("http")
https = require("https")
request = require("request")
Promise = require("bluebird")
sempahore = require("semaphore")
CA = require("../../lib/ca")
Promise.promisifyAll(fs)
ca = null
httpsSrv = null
httpsPort = null
sslServers = {}
sslSemaphores = {}
onClientError = (err) ->
console.log "CLIENT ERROR", err
onError = (err) ->
console.log "ERROR", err
onRequest = (req, res) ->
console.log "onRequest!!!!!!!!!", req.url, req.headers, req.method
hostPort = parseHostAndPort(req)
# req.pause()
opts = {
url: req.url
baseUrl: "https://" + hostPort.host + ":" + hostPort.port
method: req.method
headers: req.headers
}
req.pipe(request(opts))
.on "error", ->
console.log "**ERROR", req.url
res.statusCode = 500
res.end()
.pipe(res)
parseHostAndPort = (req, defaultPort) ->
host = req.headers.host
return null if not host
hostPort = parseHost(host, defaultPort)
## this handles paths which include the full url. This could happen if it's a proxy
if m = req.url.match(/^http:\/\/([^\/]*)\/?(.*)$/)
parsedUrl = url.parse(req.url)
hostPort.host = parsedUrl.hostname
hostPort.port = parsedUrl.port
req.url = parsedUrl.path
hostPort
parseHost = (hostString, defaultPort) ->
if m = hostString.match(/^http:\/\/(.*)/)
parsedUrl = url.parse(hostString)
return {
host: parsedUrl.hostname
port: parsedUrl.port
}
hostPort = hostString.split(':')
host = hostPort[0]
port = if hostPort.length is 2 then +hostPort[1] else defaultPort
return {
host: host
port: port
}
onConnect = (req, socket, head) ->
console.log "ON CONNECT!!!!!!!!!!!!!!!"
## tell the client that the connection is established
# socket.write('HTTP/' + req.httpVersion + ' 200 OK\r\n\r\n', 'UTF-8', function() {
# // creating pipes in both ends
# conn.pipe(socket);
# socket.pipe(conn);
# });
console.log "URL", req.url
console.log "HEADERS", req.headers
console.log "HEAD IS", head
console.log "HEAD LENGTH", head.length
# srvUrl = url.parse("http://#{req.url}")
# conn = null
# cb = ->
# socket.write('HTTP/1.1 200 Connection Established\r\n' +
# 'Proxy-agent: Cypress\r\n' +
# '\r\n')
# conn.write(head)
# conn.pipe(socket)
# socket.pipe(conn)
# conn = net.connect(srvUrl.port, srvUrl.hostname, cb)
# conn.on "error", (err) ->
# ## TODO: attach error handling here
# console.log "*******ERROR CONNECTING", err, err.stack
# # conn.on "close", ->
# # console.log "CONNECTION CLOSED", arguments
# return
# URL www.cypress.io:443
# HEADERS { host: 'www.cypress.io:443',
# 'proxy-connection': 'keep-alive',
# 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2609.0 Safari/537.36' }
# HEAD IS <Buffer >
# HEAD LENGTH 0
getHttpsServer = (hostname) ->
onCertificateRequired(hostname)
.then (certPaths) ->
Promise.props({
keyFileExists: fs.statAsync(certPaths.keyFile)
certFileExists: fs.statAsync(certPaths.certFile)
})
.catch (err) ->
onCertificateMissing(certPaths)
.then (data = {}) ->
return {
key: data.keyFileData
cert: data.certFileData
hosts: data.hosts
}
.then (data = {}) ->
hosts = [hostname]
delete data.hosts
hosts.forEach (host) ->
console.log "ADD CONTEXT", host, data
httpsSrv.addContext(host, data)
# sslServers[host] = { port: httpsPort }
# return cb(null, self.httpsPort)
return httpsPort
onCertificateMissing = (certPaths) ->
hosts = certPaths.hosts #or [ctx.hostname]
ca.generateServerCertificateKeys(hosts)
.spread (certPEM, privateKeyPEM) ->
return {
hosts: hosts
keyFileData: privateKeyPEM
certFileData: certPEM
}
onCertificateRequired = (hostname) ->
Promise.resolve({
keyFile: ""
certFile: ""
hosts: [hostname]
})
makeConnection = (port) ->
console.log "makeConnection", port
conn = net.connect port, ->
console.log "connected to", port#, socket, conn, head
socket.pipe(conn)
conn.pipe(socket)
socket.emit("data", head)
return socket.resume()
conn.on "error", onError
onServerConnectData = (head) ->
firstBytes = head[0]
if firstBytes is 0x16 or firstBytes is 0x80 or firstBytes is 0x00
{hostname} = url.parse("http://#{req.url}")
if sslServer = sslServers[hostname]
return makeConnection(sslServer.port)
wildcardhost = hostname.replace(/[^\.]+\./, "*.")
sem = sslSemaphores[wildcardhost]
if not sem
sem = sslSemaphores[wildcardhost] = sempahore(1)
sem.take ->
leave = ->
process.nextTick ->
console.log "leaving sem"
sem.leave()
if sslServer = sslServers[hostname]
leave()
return makeConnection(sslServer.port)
if sslServer = sslServers[wildcardhost]
leave()
sslServers[hostname] = {
port: sslServer
}
return makeConnection(sslServers[hostname].port)
getHttpsServer(hostname)
.then (port) ->
leave()
makeConnection(port)
else
throw new Error("@httpPort")
makeConnection(@httpPort)
if not head or head.length is 0
socket.once "data", onConnect.bind(@, req, socket)
socket.write "HTTP/1.1 200 OK\r\n"
if req.headers["proxy-connection"] is "keep-alive"
socket.write("Proxy-Connection: keep-alive\r\n")
socket.write("Connection: keep-alive\r\n")
return socket.write("\r\n")
else
socket.pause()
onServerConnectData(head)
prx = http.createServer()
prx.on("connect", onConnect)
prx.on("request", onRequest)
prx.on("clientError", onClientError)
prx.on("error", onError)
module.exports = {
prx: prx
startHttpsSrv: ->
new Promise (resolve) ->
httpsSrv = https.createServer({})
# httpsSrv.timeout = 0
httpsSrv.on("connect", onConnect)
httpsSrv.on("request", onRequest)
httpsSrv.on("clientError", onClientError)
httpsSrv.on("error", onError)
httpsSrv.listen ->
resolve([httpsSrv.address().port, httpsSrv])
start: ->
dir = path.join(process.cwd(), "ca")
CA.create(dir)
.then (c) =>
ca = c
@startHttpsSrv()
.spread (port, httpsSrv) ->
httpsPort = port
new Promise (resolve) ->
prx.listen 3333, ->
console.log "server listening on port: 3333"
resolve(prx)
stop: ->
new Promise (resolve) ->
prx.close(resolve)
}
packages/https-proxy/test/integration/proxy_spec.coffee
+97 -3
View File
@@ -2,10 +2,12 @@ require("../spec_helper")
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0"
_ = require("lodash")
DebugProxy = require("debugging-proxy")
net = require("net")
path = require("path")
Promise = require("bluebird")
proxy = require("../helpers/proxy")
mitmProxy = require("../helpers/mitm")
httpServer = require("../helpers/http_server")
httpsServer = require("../helpers/https_server")
@@ -14,8 +16,6 @@ describe "Proxy", ->
Promise.join(
httpServer.start()
# mitmProxy.start()
httpsServer.start(8443)
httpsServer.start(8444)
@@ -91,6 +91,25 @@ describe "Proxy", ->
.then (html) ->
expect(html).to.include("https server")
it "closes outgoing connections when client disconnects", ->
@sandbox.spy(net.Socket.prototype, 'connect')
request({
strictSSL: false
url: "https://localhost:8444/replace"
proxy: "http://localhost:3333"
resolveWithFullResponse: true
})
.then (res) =>
## ensure client has disconnected
expect(res.socket.destroyed).to.be.true
## ensure the outgoing socket created for this connection was destroyed
socket = net.Socket.prototype.connect.getCalls()
.find (call) =>
_.isEqual(call.args.slice(0,2), ["8444", "localhost"])
.thisValue
expect(socket.destroyed).to.be.true
it "can boot the httpServer", ->
request({
strictSSL: false
@@ -139,3 +158,78 @@ describe "Proxy", ->
url: "https://localhost:8443/"
proxy: "http://localhost:3333"
})
context "with an upstream proxy", ->
beforeEach ->
@oldEnv = Object.assign({}, process.env)
process.env.NO_PROXY = ""
process.env.HTTP_PROXY = process.env.HTTPS_PROXY = "http://localhost:9001"
@upstream = new DebugProxy({
keepRequests: true
})
@upstream.start(9001)
it "passes a request to an https server through the upstream", ->
request({
strictSSL: false
url: "https://localhost:8444/"
proxy: "http://localhost:3333"
}).then (res) =>
expect(@upstream.getRequests()[0]).to.include({
url: 'localhost:8444'
https: true
})
expect(res).to.contain("https server")
it "uses HTTP basic auth when provided", ->
@upstream.setAuth({
username: 'foo'
password: 'bar'
})
process.env.HTTP_PROXY = process.env.HTTPS_PROXY = "http://foo:bar@localhost:9001"
request({
strictSSL: false
url: "https://localhost:8444/"
proxy: "http://localhost:3333"
}).then (res) =>
expect(@upstream.getRequests()[0]).to.include({
url: 'localhost:8444'
https: true
})
expect(res).to.contain("https server")
it "closes outgoing connections when client disconnects", ->
@sandbox.spy(net.Socket.prototype, 'connect')
request({
strictSSL: false
url: "https://localhost:8444/replace"
proxy: "http://localhost:3333"
resolveWithFullResponse: true
forever: false
})
.then (res) =>
## ensure client has disconnected
expect(res.socket.destroyed).to.be.true
## ensure the outgoing socket created for this connection was destroyed
socket = net.Socket.prototype.connect.getCalls()
.find (call) =>
_.isEqual(call.args[0][0], {
host: 'localhost'
port: 9001
})
.thisValue
new Promise (resolve) ->
socket.on 'close', =>
expect(socket.destroyed).to.be.true
resolve()
afterEach ->
@upstream.stop()
Object.assign(process.env, @oldEnv)
packages/https-proxy/test/mocha.opts
+1 -1
View File
@@ -1,5 +1,5 @@
test/unit
test/integration
--reporter spec
--compilers coffee:@packages/coffee/register
--compilers ts:@packages/ts/register,coffee:@packages/coffee/register
--recursive
packages/https-proxy/test/unit/server_spec.coffee
+5 -2
View File
@@ -1,5 +1,6 @@
require("../spec_helper")
EE = require("events")
Promise = require("bluebird")
proxy = require("../helpers/proxy")
Server = require("../../lib/server")
@@ -35,7 +36,7 @@ describe "lib/server", ->
it "calls options.onError with err and port", (done) ->
onError = @sandbox.stub()
socket = {}
socket = new EE()
head = {}
@setup({onError: onError})
@@ -46,7 +47,9 @@ describe "lib/server", ->
err = onError.getCall(0).args[0]
expect(err.message).to.eq("connect ECONNREFUSED 127.0.0.1:8444")
expect(onError).to.be.calledWithMatch(err, socket, head, 8444)
expect(onError.getCall(0).args[1]).to.eq(socket)
expect(onError.getCall(0).args[2]).to.eq(head)
expect(onError.getCall(0).args[3]).to.eq("8444")
done()
return