Files
cypress/packages/https-proxy/test/integration/proxy_spec.coffee
Zach Bloomquist b8a6baaca3 Retry certain requests on failure (#4015)
* retry requests, basic e2e test

* basic e2e test for chrome behavior

* don't use timeouts to test

* some minor cleanup

* validate google chrome's retry behavior w/ a proxy

* get retries on network errors workin

* cleanup

* final changes before switching approach

* Reverting previous approach in request.coffee, server.coffee

* add retryOnNetworkFailure

* now works with retryOnStatusCodeFailure

* retry 4 times in addition to the 1 initial attempt

* add tests for subresources

* much improved error handling

* have the e2e test really work

* e2e baseurl check

* retry baseurl check

* remove special handling for node 8.2.1 bug

* WIP: continue making progress building out request retry

- swap out passthrough’s for pumpify / duplexify / pump
- clean up error handling / messaging / retry logic

* pipe clientrequest events

* buffer req body to disk, restore error handling/retrying

* don't bubble up errors from failed attempts

* actually pipe reqstream, oops

* add some e2e tests for request body

* revert lib/request.coffee to 7be1051

* add almost-working lazy-stream

* manually fire the 'pipe' event on the reqStream to copy headers to the outgoing message

- restore the ‘error’ propagation so that all tests pass for now

* cleanup leaking 'undefined' into stdout, causing failing e2e tests

- do not set onWarning as a property of project, just pass as an
argument

* add new options to request_spec, deduplicate default opts

* use stream buffer in request.coffee

* revert request.coffee

* last stream_buffer changes before removing fs stuff

* remove fs stuff from stream_buffer, add stream piping tests

* it works! :tada::tada:🎉 using duplexify to separate delayStream and reqBodyBuffer

* retry for .5s max on ECONNREFUSED

* add error codes

* don't timeout proxied requests

* restore baseurl check

* update new e2e tests

* make delay work with rp

* propagate clientresponse events

* removing tests that don't do anything now that we don't ESOCKETTIMEOUT on proxied requests

* add new visit, req options to index.d.ts

* don't fail on server-performance-test

* make retries with status code work again

* account for different stack trace in ci

* fix test

* retry https requests

* add tests for https passthru retries working

* clean up error handling for https-proxy

* fix failing https-proxy tests, tweak agent error handling to prevent multiple callbacks

* make expectedExitCode actual vs. expected in the correct order

* bump up e2e test timeout so it can retry and still work

* update tests

* retry up to 500ms on proxied requests

* add tests for incompatible options errors

* remove .only

* maybe this will help it act more consistently

* help e2e test work in ci

* don't reconnect on already made connections

* clarify naming

* wip: testing https proxy

* better debug calls

* WIP: getting proxy passthrough retry tests going

* handle retrying upstream proxy errors

- add tests for successfully retrying proxy errors and for unsuccessful
retries
- fix onClose errors when proxy connection is closed before enough data
is received
- fix not returning setTimeout correctly

* group related code accordingly

* do not build typescript by default, drop extension from main

* more TODO notes

* don't set a default NO_PROXY if NO_PROXY = ''

* debugging-proxy@2.0.0

* null out reqBodyBuffer when req finishes

* wip: retry in agent, not https-proxy [skip-ci]

* update https-proxy to use @packages/network retries

* retry after connection but before proxy tunnel established

* appease my linty overlords

* update https-proxy tests

* update agent specs, decided to still use tls.connect

it's easier to test and has less complexity

* test retrying HTTPS passthru

* debugging-proxy@2.0.1

* increase defaultCommandTimeout 100 -> 200 to prevent flake in CI

* auto formatting

* fix test to be dynamic and not rely on magic constants

* copy types field when linking proxy images, update packages/network types field

* linting

* add network index.js file

* linting

* improve error messaging experience when verifying base url

* only insert 1 new line

* fix failing test not binding to localhost

* removed test that's covered by e2e specs

* remove dash in 're-try'

* some cleanup for readability

* use allocUnsafe per perf

* unset NO_PROXY with an empty string

* move retry ensuring the baseUrl into url, cleanup some imperative code

* if the head is already present during connect, make connection, else wait for first head bytes

* minor formatting, clarity around conditions, naming

* rename retryInterval -> retryIntervals

* set defaults for requests during creation for clarity

* rename send -> sendPromise to pair better with sendStream

* use retryIntervals instead of juggling MAX_REQUEST_RETRIES

- ensure debug messages are consistent between request streams +
promises
- set static constants

* DRY up status check + network failure + retry logic

- keeps the debug logic identical between promises + streams
- ensures all logic paths are also consistent
- consolidates the pop’ing of intervals in a single place

* find / replace fail

* derp

* make the logic actually correct, set intervals as cloned defaults for recursive lookup

* pass arg correctly

* reduce debugging noise, formatting

* rename intervals -> delaysRemaining for clarity

* added unit tests around getDelayForRetry

* set retryIntervals as default options correctly, add unit tests


Co-authored-by: Brian Mann <brian.mann86@gmail.com>
Co-authored-by: Jennifer Shehane <jennifer@cypress.io>
Co-authored-by: Gleb Bahmutov <gleb.bahmutov@gmail.com>
2019-05-15 12:46:55 -04:00

249 lines
6.4 KiB
CoffeeScript

require("../spec_helper")
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0"
_ = require("lodash")
DebugProxy = require("@cypress/debugging-proxy")
net = require("net")
path = require("path")
Promise = require("bluebird")
proxy = require("../helpers/proxy")
httpServer = require("../helpers/http_server")
httpsServer = require("../helpers/https_server")
describe "Proxy", ->
beforeEach ->
Promise.join(
httpServer.start()
httpsServer.start(8443)
httpsServer.start(8444)
proxy.start(3333)
.then (@proxy) =>
)
afterEach ->
Promise.join(
httpServer.stop()
httpsServer.stop()
proxy.stop()
)
it "can request the googles", ->
## give some padding to external
## network request
@timeout(10000)
Promise.all([
request({
strictSSL: false
proxy: "http://localhost:3333"
url: "https://www.google.com"
})
request({
strictSSL: false
proxy: "http://localhost:3333"
url: "https://mail.google.com"
})
request({
strictSSL: false
proxy: "http://localhost:3333"
url: "https://google.com"
})
])
it "can call the httpsDirectly without a proxy", ->
request({
strictSSL: false
url: "https://localhost:8443"
})
it "can boot the httpsServer", ->
request({
strictSSL: false
url: "https://localhost:8443/"
proxy: "http://localhost:3333"
})
.then (html) ->
expect(html).to.include("https server")
it "yields the onRequest callback", ->
request({
strictSSL: false
url: "https://localhost:8443/replace"
proxy: "http://localhost:3333"
})
.then (html) ->
expect(html).to.include("replaced content")
it "can pass directly through", ->
## this will fail due to dynamic cert
## generation when strict ssl is true
request({
strictSSL: false
url: "https://localhost:8444/replace"
proxy: "http://localhost:3333"
})
.then (html) ->
expect(html).to.include("https server")
it "retries 5 times", ->
@sandbox.spy(net, 'connect')
request({
strictSSL: false
url: "https://localhost:12344"
proxy: "http://localhost:3333"
})
.then ->
throw new Error("should not reach")
.catch ->
expect(net.connect).to.have.callCount(5)
it "closes outgoing connections when client disconnects", ->
@sandbox.spy(net, 'connect')
request({
strictSSL: false
url: "https://localhost:8444/replace"
proxy: "http://localhost:3333"
resolveWithFullResponse: true
})
.then (res) =>
## ensure client has disconnected
expect(res.socket.destroyed).to.be.true
## ensure the outgoing socket created for this connection was destroyed
socket = net.connect.getCalls()
.find (call) =>
call.args[0].port == "8444" && call.args[0].host == "localhost"
.returnValue
expect(socket.destroyed).to.be.true
it "can boot the httpServer", ->
request({
strictSSL: false
url: "http://localhost:8080/"
proxy: "http://localhost:3333"
})
.then (html) ->
expect(html).to.include("http server")
context "generating certificates", ->
it "reuses existing certificates", ->
request({
strictSSL: false
url: "https://localhost:8443/"
proxy: "http://localhost:3333"
})
.then =>
proxy.reset()
## force this to reject if its called
@sandbox.stub(@proxy, "_generateMissingCertificates").rejects(new Error("should not call"))
request({
strictSSL: false
url: "https://localhost:8443/"
proxy: "http://localhost:3333"
})
context "closing", ->
it "resets sslServers and can reopen", ->
request({
strictSSL: false
url: "https://localhost:8443/"
proxy: "http://localhost:3333"
})
.then =>
proxy.stop()
.then =>
proxy.start(3333)
.then =>
## force this to reject if its called
@sandbox.stub(@proxy, "_generateMissingCertificates").rejects(new Error("should not call"))
request({
strictSSL: false
url: "https://localhost:8443/"
proxy: "http://localhost:3333"
})
context "with an upstream proxy", ->
beforeEach ->
process.env.NO_PROXY = ""
process.env.HTTP_PROXY = process.env.HTTPS_PROXY = "http://localhost:9001"
@upstream = new DebugProxy({
keepRequests: true
})
@upstream.start(9001)
it "passes a request to an https server through the upstream", ->
@upstream._onConnect = (domain, port) ->
expect(domain).to.eq('localhost')
expect(port).to.eq('8444')
return true
request({
strictSSL: false
url: "https://localhost:8444/"
proxy: "http://localhost:3333"
}).then (res) =>
expect(res).to.contain("https server")
it "uses HTTP basic auth when provided", ->
@upstream.setAuth({
username: 'foo'
password: 'bar'
})
@upstream._onConnect = (domain, port) ->
expect(domain).to.eq('localhost')
expect(port).to.eq('8444')
return true
process.env.HTTP_PROXY = process.env.HTTPS_PROXY = "http://foo:bar@localhost:9001"
request({
strictSSL: false
url: "https://localhost:8444/"
proxy: "http://localhost:3333"
}).then (res) =>
expect(res).to.contain("https server")
it "closes outgoing connections when client disconnects", ->
@sandbox.spy(net, 'connect')
request({
strictSSL: false
url: "https://localhost:8444/replace"
proxy: "http://localhost:3333"
resolveWithFullResponse: true
forever: false
})
.then (res) =>
## ensure client has disconnected
expect(res.socket.destroyed).to.be.true
## ensure the outgoing socket created for this connection was destroyed
socket = net.connect.getCalls()
.find (call) =>
call.args[0].port == 9001 && call.args[0].host == "localhost"
.returnValue
new Promise (resolve) ->
socket.on 'close', =>
expect(socket.destroyed).to.be.true
resolve()
afterEach ->
@upstream.stop()
delete process.env.HTTP_PROXY
delete process.env.HTTPS_PROXY
delete process.env.NO_PROXY