mirror of
https://github.com/cypress-io/cypress.git
synced 2026-01-05 14:09:46 -06:00
chore: move server e2e tests to system-tests (#16354)
Co-authored-by: Brian Mann <brian.mann86@gmail.com> Co-authored-by: Zach Bloomquist <git@chary.us> Co-authored-by: Zach Bloomquist <github@chary.us>
This commit is contained in:
Jessica Sachs
@@ -14,6 +14,21 @@
|
||||
**/support/fixtures/projects/**/static/*
|
||||
**/support/fixtures/projects/**/*.jsx
|
||||
**/support/fixtures/projects/**/fail.js
|
||||
|
||||
system-tests/fixtures/*
|
||||
!system-tests/projects
|
||||
system-tests/projects/**/_fixtures/*
|
||||
system-tests/projects/**/static/*
|
||||
system-tests/projects/**/*.jsx
|
||||
system-tests/projects/**/fail.js
|
||||
system-tests/lib/scaffold/plugins/index.js
|
||||
system-tests/lib/scaffold/support/index.js
|
||||
system-tests/lib/scaffold/support/commands.js
|
||||
system-tests/test/support/projects/e2e/cypress/
|
||||
system-tests/projects/e2e/cypress/integration/stdout_exit_early_failing_spec.js
|
||||
system-tests/projects/e2e/cypress/integration/typescript_syntax_error_spec.ts
|
||||
|
||||
|
||||
**/test/fixtures
|
||||
**/vendor
|
||||
|
||||
@@ -23,11 +38,7 @@ cli/types
|
||||
packages/example
|
||||
|
||||
packages/extension/test/helpers/background.js
|
||||
packages/server/lib/scaffold/plugins/index.js
|
||||
packages/server/lib/scaffold/support/index.js
|
||||
packages/server/lib/scaffold/support/commands.js
|
||||
packages/server/test/support/fixtures/projects/e2e/cypress/integration/stdout_exit_early_failing_spec.js
|
||||
packages/server/test/support/fixtures/projects/e2e/cypress/integration/typescript_syntax_error_spec.ts
|
||||
integration/stdout_exit_early_failing_spec.js
|
||||
|
||||
npm/webpack-preprocessor/cypress/tests/e2e/compile-error.js
|
||||
npm/webpack-preprocessor/examples/use-babelrc/cypress/integration/spec.js
|
||||
|
||||
4
.gitignore
vendored
4
.gitignore
vendored
@@ -40,6 +40,10 @@ packages/server/support
|
||||
packages/server/test/support/fixtures/server/imgs
|
||||
packages/server/test/support/fixtures/server/libs
|
||||
|
||||
# from system-tests
|
||||
system-tests/.projects
|
||||
system-tests/fixtures/large-img
|
||||
|
||||
# from npm/react
|
||||
/npm/react/bin/*
|
||||
/npm/react/cypress/videos
|
||||
|
||||
14
.vscode/terminals.json
vendored
14
.vscode/terminals.json
vendored
@@ -24,12 +24,20 @@
|
||||
"command": "yarn cypress:run --project ../project"
|
||||
},
|
||||
{
|
||||
"name": "packages/server test-e2e",
|
||||
"name": "cypress open (CT)",
|
||||
"focus": true,
|
||||
"onlySingle": true,
|
||||
"execute": true,
|
||||
"cwd": "[cwd]/packages/server-ct",
|
||||
"command": "yarn cypress:open"
|
||||
},
|
||||
{
|
||||
"name": "system-tests test",
|
||||
"focus": true,
|
||||
"onlySingle": true,
|
||||
"execute": false,
|
||||
"cwd": "[cwd]/packages/server",
|
||||
"command": "yarn test-e2e [fileBasename]"
|
||||
"cwd": "[cwd]/system-tests",
|
||||
"command": "yarn test [fileBasename]"
|
||||
},
|
||||
{
|
||||
"name": "packages/server test-watch",
|
||||
|
||||
@@ -396,6 +396,7 @@ By default, top level tasks will execute for all packages. However, most scripts
|
||||
| `test-unit` | Run unit tests |
|
||||
| `test-integration` | Run integration tests |
|
||||
| `test-e2e` | Run end-to-end tests |
|
||||
| `test-system` | Run system tests |
|
||||
| `test-watch` | Run unit tests and rebuild/rerun on file changes |
|
||||
|
||||
> Most of the time you will only want to run a task within a specific package; this can be done by providing the package name as a scope to the top level task.
|
||||
@@ -428,7 +429,6 @@ Each package is responsible for building itself and testing itself and can do so
|
||||
| `test` | Runs all tests once (this usually means running unit tests; via `yarn test-unit`) |
|
||||
| `test-unit` | Run all unit tests within the package; `exit 0` if N/A |
|
||||
| `test-integration` | Run all integration tests within the package; `exit 0` if N/A |
|
||||
| `test-e2e` | Run all e2e tests within the package; `exit 0` if N/A |
|
||||
| `test-watch` | Run all unit tests in the package in watch mode |
|
||||
|
||||
#### Debugging
|
||||
@@ -486,12 +486,12 @@ This is to ensure that links do not go dead in older versions of Cypress when th
|
||||
|
||||
For most packages there are typically unit and integration tests.
|
||||
|
||||
Our true e2e tests are in [`packages/server`](packages/server), which test the full stack all together.
|
||||
Please refer to each packages' `README.md` which documents how to run tests. It is not feasible to try to run all of the tests together. We run our entire test fleet across over a dozen containers in CI.
|
||||
|
||||
There are also a set of system tests in [`system-tests`](system-tests) which attempt to test the entire Cypress App as close to real world as possible. See the [`README`](system-tests/README.md) for more information.
|
||||
|
||||
Additionally, we test the code by running it against various other example projects in CI. See CI badges and links at the top of this document.
|
||||
|
||||
Please refer to each packages' `README.md` which documents how to run tests. It is not feasible to try to run all of the tests together. We run our entire test fleet across over a dozen containers in CI.
|
||||
|
||||
If you're curious how we manage all of these tests in CI check out our [`circle.yml`](circle.yml) file found in the root `cypress` directory.
|
||||
|
||||
#### Docker
|
||||
|
||||
42
circle.yml
42
circle.yml
@@ -112,6 +112,7 @@ commands:
|
||||
mkdir -p /tmp/node_modules_cache
|
||||
mv ~/cypress/node_modules /tmp/node_modules_cache/root_node_modules
|
||||
mv ~/cypress/cli/node_modules /tmp/node_modules_cache/cli_node_modules
|
||||
mv ~/cypress/system-tests/node_modules /tmp/node_modules_cache/system-tests_node_modules
|
||||
mv ~/cypress/globbed_node_modules /tmp/node_modules_cache/globbed_node_modules
|
||||
|
||||
build-and-persist:
|
||||
@@ -142,6 +143,7 @@ commands:
|
||||
if [[ -d "/tmp/node_modules_cache" ]]; then
|
||||
mv /tmp/node_modules_cache/root_node_modules ~/cypress/node_modules
|
||||
mv /tmp/node_modules_cache/cli_node_modules ~/cypress/cli/node_modules
|
||||
mv /tmp/node_modules_cache/system-tests_node_modules ~/cypress/system-tests/node_modules
|
||||
mv /tmp/node_modules_cache/globbed_node_modules ~/cypress/globbed_node_modules
|
||||
rm -rf /tmp/node_modules_cache
|
||||
fi
|
||||
@@ -194,6 +196,7 @@ commands:
|
||||
paths:
|
||||
- node_modules
|
||||
- cli/node_modules
|
||||
- system-tests/node_modules
|
||||
- globbed_node_modules
|
||||
- unless:
|
||||
condition: <<parameters.is-mac>>
|
||||
@@ -417,7 +420,7 @@ commands:
|
||||
path: ./packages/runner-ct/cypress/videos
|
||||
- store-npm-logs
|
||||
|
||||
run-e2e-tests:
|
||||
run-system-tests:
|
||||
parameters:
|
||||
browser:
|
||||
description: browser shortname to target
|
||||
@@ -425,8 +428,9 @@ commands:
|
||||
steps:
|
||||
- restore_cached_workspace
|
||||
- run:
|
||||
name: Run system tests
|
||||
command: |
|
||||
ALL_SPECS=`circleci tests glob "/root/cypress/packages/server/test/e2e/*spec*"`
|
||||
ALL_SPECS=`circleci tests glob "/root/cypress/system-tests/test/*spec*"`
|
||||
SPECS=
|
||||
for file in $ALL_SPECS; do
|
||||
# filter out non_root tests, they have their own stage
|
||||
@@ -438,7 +442,7 @@ commands:
|
||||
done
|
||||
SPECS=`echo $SPECS | xargs -n 1 | circleci tests split --split-by=timings`
|
||||
echo SPECS=$SPECS
|
||||
yarn workspace @packages/server test $SPECS --browser <<parameters.browser>>
|
||||
yarn workspace @tooling/system-tests test:ci $SPECS --browser <<parameters.browser>>
|
||||
- verify-mocha-results
|
||||
- store_test_results:
|
||||
path: /tmp/cypress
|
||||
@@ -1080,37 +1084,37 @@ jobs:
|
||||
path: /tmp/artifacts
|
||||
- store-npm-logs
|
||||
|
||||
server-e2e-tests-chrome:
|
||||
system-tests-chrome:
|
||||
<<: *defaults
|
||||
resource_class: medium
|
||||
parallelism: 8
|
||||
steps:
|
||||
- run-e2e-tests:
|
||||
- run-system-tests:
|
||||
browser: chrome
|
||||
|
||||
server-e2e-tests-electron:
|
||||
system-tests-electron:
|
||||
<<: *defaults
|
||||
resource_class: medium
|
||||
parallelism: 8
|
||||
steps:
|
||||
- run-e2e-tests:
|
||||
- run-system-tests:
|
||||
browser: electron
|
||||
|
||||
server-e2e-tests-firefox:
|
||||
system-tests-firefox:
|
||||
<<: *defaults
|
||||
resource_class: medium
|
||||
parallelism: 8
|
||||
steps:
|
||||
- run-e2e-tests:
|
||||
- run-system-tests:
|
||||
browser: firefox
|
||||
|
||||
server-e2e-tests-non-root:
|
||||
system-tests-non-root:
|
||||
<<: *defaults
|
||||
resource_class: medium
|
||||
steps:
|
||||
- restore_cached_workspace
|
||||
- run:
|
||||
command: yarn workspace @packages/server test ./test/e2e/non_root*spec* --browser electron
|
||||
command: yarn workspace @tooling/system-tests test:ci "test/non_root*spec*" --browser electron
|
||||
- verify-mocha-results
|
||||
- store_test_results:
|
||||
path: /tmp/cypress
|
||||
@@ -2007,16 +2011,16 @@ linux-workflow: &linux-workflow
|
||||
- server-performance-tests:
|
||||
requires:
|
||||
- build
|
||||
- server-e2e-tests-chrome:
|
||||
- system-tests-chrome:
|
||||
requires:
|
||||
- build
|
||||
- server-e2e-tests-electron:
|
||||
- system-tests-electron:
|
||||
requires:
|
||||
- build
|
||||
- server-e2e-tests-firefox:
|
||||
- system-tests-firefox:
|
||||
requires:
|
||||
- build
|
||||
- server-e2e-tests-non-root:
|
||||
- system-tests-non-root:
|
||||
executor: non-root-docker-user
|
||||
requires:
|
||||
- build
|
||||
@@ -2123,10 +2127,10 @@ linux-workflow: &linux-workflow
|
||||
- driver-integration-tests-firefox
|
||||
- driver-integration-tests-chrome
|
||||
- driver-integration-tests-electron
|
||||
- server-e2e-tests-non-root
|
||||
- server-e2e-tests-firefox
|
||||
- server-e2e-tests-electron
|
||||
- server-e2e-tests-chrome
|
||||
- system-tests-non-root
|
||||
- system-tests-firefox
|
||||
- system-tests-electron
|
||||
- system-tests-chrome
|
||||
- server-performance-tests
|
||||
- server-integration-tests
|
||||
- server-unit-tests
|
||||
|
||||
@@ -3,7 +3,8 @@
|
||||
"packages": [
|
||||
"cli",
|
||||
"packages/*",
|
||||
"npm/*"
|
||||
"npm/*",
|
||||
"system-tests"
|
||||
],
|
||||
"useWorkspaces": true,
|
||||
"version": "0.0.0"
|
||||
|
||||
@@ -268,6 +268,11 @@ const preprocessor: WebpackPreprocessor = (options: PreprocessorOptions = {}): F
|
||||
|
||||
const jsonStats = stats.toJson()
|
||||
|
||||
// these stats are really only useful for debugging
|
||||
if (jsonStats.warnings.length > 0) {
|
||||
debug(`warnings for ${outputPath} %o`, jsonStats.warnings)
|
||||
}
|
||||
|
||||
if (stats.hasErrors()) {
|
||||
err = new Error('Webpack Compilation Error')
|
||||
|
||||
@@ -279,17 +284,11 @@ const preprocessor: WebpackPreprocessor = (options: PreprocessorOptions = {}): F
|
||||
|
||||
err.message += `\n${errorsToAppend}`
|
||||
|
||||
debug('stats had error(s)')
|
||||
debug('stats had error(s) %o', jsonStats.errors)
|
||||
|
||||
return rejectWithErr(err)
|
||||
}
|
||||
|
||||
// these stats are really only useful for debugging
|
||||
if (jsonStats.warnings.length > 0) {
|
||||
debug(`warnings for ${outputPath}`)
|
||||
debug(jsonStats.warnings)
|
||||
}
|
||||
|
||||
debug('finished bundling', outputPath)
|
||||
if (debugStats.enabled) {
|
||||
/* eslint-disable-next-line no-console */
|
||||
|
||||
@@ -83,7 +83,7 @@ exports.runTest = async (options = {}) => {
|
||||
VIDEO_COMPRESSION_THROTTLE: 120000,
|
||||
|
||||
// don't fail our own tests running from forked PR's
|
||||
CYPRESS_INTERNAL_E2E_TESTS: '1',
|
||||
CYPRESS_INTERNAL_SYSTEM_TESTS: '1',
|
||||
CYPRESS_ENV: 'test',
|
||||
})
|
||||
|
||||
|
||||
@@ -49,12 +49,12 @@
|
||||
"test": "yarn lerna exec yarn test --scope cypress --scope \"'@packages/{electron,extension,https-proxy,launcher,net-stubbing,network,proxy,rewriter,runner,runner-shared,socket}'\"",
|
||||
"test-debug": "lerna exec yarn test-debug --ignore \"'@packages/{desktop-gui,driver,root,static,web-config}'\"",
|
||||
"pretest-e2e": "yarn ensure-deps",
|
||||
"test-e2e": "lerna exec yarn test-e2e --ignore \"'@packages/{desktop-gui,driver,root,static,web-config}'\"",
|
||||
"test-integration": "lerna exec yarn test-integration --ignore \"'@packages/{desktop-gui,driver,root,static,web-config}'\"",
|
||||
"test-mocha": "mocha --reporter spec scripts/spec.js",
|
||||
"test-mocha-snapshot": "mocha scripts/mocha-snapshot-spec.js",
|
||||
"test-npm-package-release-script": "npx lerna exec --scope \"@cypress/*\" -- npx --no-install semantic-release --dry-run",
|
||||
"test-s3-api": "node -r ./packages/ts/register scripts/binary/s3-api-demo.ts",
|
||||
"test-system": "yarn workspace @tooling/system-tests test",
|
||||
"test-scripts": "mocha -r packages/ts/register --reporter spec 'scripts/unit/**/*spec.js'",
|
||||
"test-scripts-watch": "yarn test-scripts --watch --watch-extensions 'ts,js'",
|
||||
"pretest-unit": "yarn ensure-deps",
|
||||
@@ -224,7 +224,8 @@
|
||||
"packages": [
|
||||
"cli",
|
||||
"packages/*",
|
||||
"npm/*"
|
||||
"npm/*",
|
||||
"system-tests"
|
||||
],
|
||||
"nohoist": [
|
||||
"**/@ffmpeg-installer",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import './setup'
|
||||
|
||||
const outsideError = require('../../../../server/test/support/fixtures/projects/todos/throws-error')
|
||||
const outsideError = require('@tooling/system-tests/projects/todos/throws-error')
|
||||
|
||||
describe('exception failures', () => {
|
||||
it('in spec file', () => {
|
||||
|
||||
@@ -39,7 +39,6 @@ yarn workspace @packages/server build-prod
|
||||
* `yarn test-unit` executes unit tests in [`test/unit`](./test/unit)
|
||||
* `yarn test-integration` executes integration tests in [`test/integration`](./test/integration)
|
||||
* `yarn test-performance` executes performance tests in [`test/performance`](./test/performance)
|
||||
* `yarn test-e2e` executes the large (slow) end to end tests in [`test/e2e`](./test/e2e)
|
||||
|
||||
You can also use the `test-watch` command to rerun a test file whenever there is a change:
|
||||
|
||||
@@ -47,8 +46,6 @@ You can also use the `test-watch` command to rerun a test file whenever there is
|
||||
yarn test-watch /test/path/to/spec.js
|
||||
```
|
||||
|
||||
When running e2e tests, some test projects output verbose logs. To see them run the test with `DEBUG=cypress:e2e` environment variable.
|
||||
|
||||
### Running individual unit tests
|
||||
|
||||
```bashtest-kitchensink
|
||||
@@ -67,19 +64,9 @@ yarn test test/integration/cli_spec.js
|
||||
yarn test-integration cli_spec ## shorthand, uses globbing to find spec
|
||||
```
|
||||
|
||||
### Running individual e2e tests
|
||||
### Running e2e/system tests
|
||||
|
||||
```bash
|
||||
yarn test <path/to/test>
|
||||
yarn test test/e2e/1_async_timeouts_spec.js
|
||||
## or
|
||||
yarn test-e2e async_timeouts ## shorthand, uses globbing to find spec
|
||||
```
|
||||
|
||||
To keep the browser open after a spec run (for easier debugging and iterating on specs), you can pass the `--no-exit` flag to the e2e test command. Live reloading due to spec changes should also work:
|
||||
```sh
|
||||
yarn test test/e2e/go_spec.js --browser chrome --no-exit
|
||||
```
|
||||
> With the addition of Component Testing, `e2e` tests have been renamed to `system-tests` and moved to the [`system-tests`](../../system-tests) directory.
|
||||
|
||||
### Updating snaphots
|
||||
|
||||
@@ -88,5 +75,4 @@ Prepend `SNAPSHOT_UPDATE=1` to any test command. See [`snap-shot-it` instruction
|
||||
```bash
|
||||
SNAPSHOT_UPDATE=1 yarn test test/unit/api_spec.js
|
||||
SNAPSHOT_UPDATE=1 yarn test test/integration/cli_spec.js
|
||||
SNAPSHOT_UPDATE=1 yarn test-e2e async_timeout
|
||||
```
|
||||
|
||||
@@ -32,7 +32,7 @@ const logException = (err) => {
|
||||
// dont yell about any errors either
|
||||
|
||||
const runningInternalTests = () => {
|
||||
return env.get('CYPRESS_INTERNAL_E2E_TESTS') === '1'
|
||||
return env.get('CYPRESS_INTERNAL_SYSTEM_TESTS') === '1'
|
||||
}
|
||||
|
||||
const haveProjectIdAndKeyButNoRecordOption = (projectId, options) => {
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
require('./environment')
|
||||
|
||||
const _ = require('lodash')
|
||||
const path = require('path')
|
||||
const repl = require('repl')
|
||||
const history = require('repl.history')
|
||||
const browsers = require('./browsers')
|
||||
const Fixtures = require('../test/support/helpers/fixtures')
|
||||
|
||||
const replServer = repl.start({
|
||||
prompt: '> ',
|
||||
})
|
||||
|
||||
let setContext
|
||||
|
||||
// preserve the repl history
|
||||
history(replServer, path.join(process.env.HOME, '.node_history'))
|
||||
|
||||
const req = replServer.context.require
|
||||
|
||||
const getObj = function () {
|
||||
const deploy = require('../deploy')
|
||||
|
||||
return {
|
||||
lodash: _,
|
||||
deploy,
|
||||
darwin: deploy.getPlatform('darwin'),
|
||||
linux: deploy.getPlatform('linux'),
|
||||
Fixtures,
|
||||
browsers,
|
||||
|
||||
reload () {
|
||||
let key
|
||||
|
||||
for (key in require.cache) {
|
||||
delete require.cache[key]
|
||||
}
|
||||
|
||||
for (key in req.cache) {
|
||||
delete req.cache[key]
|
||||
}
|
||||
|
||||
return setContext()
|
||||
},
|
||||
|
||||
r (file) {
|
||||
return require(file)
|
||||
},
|
||||
}
|
||||
};
|
||||
|
||||
(setContext = () => {
|
||||
return _.extend(replServer.context, getObj())
|
||||
})()
|
||||
@@ -13,7 +13,6 @@
|
||||
"repl": "node repl.js",
|
||||
"start": "node ../../scripts/cypress open --dev --global",
|
||||
"test": "node ./test/scripts/run.js",
|
||||
"test-e2e": "node ./test/scripts/run.js --glob-in-dir=test/e2e",
|
||||
"test-integration": "node ./test/scripts/run.js --glob-in-dir=test/integration",
|
||||
"test-performance": "node ./test/scripts/run.js --glob-in-dir=test/performance",
|
||||
"test-unit": "node ./test/scripts/run.js --glob-in-dir=test/unit",
|
||||
@@ -143,6 +142,7 @@
|
||||
"@packages/root": "0.0.0-development",
|
||||
"@packages/socket": "0.0.0-development",
|
||||
"@packages/ts": "0.0.0-development",
|
||||
"@tooling/system-tests": "0.0.0-development",
|
||||
"@types/chai-as-promised": "7.1.2",
|
||||
"@types/chrome": "0.0.101",
|
||||
"@types/http-proxy": "1.17.4",
|
||||
|
||||
@@ -1,2 +0,0 @@
|
||||
require('@packages/ts/register')
|
||||
require('./lib/repl')
|
||||
@@ -9,7 +9,7 @@ const http = require('http')
|
||||
const Promise = require('bluebird')
|
||||
const electron = require('electron')
|
||||
const commitInfo = require('@cypress/commit-info')
|
||||
const Fixtures = require('../support/helpers/fixtures')
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
const snapshot = require('snap-shot-it')
|
||||
const stripAnsi = require('strip-ansi')
|
||||
const debug = require('debug')('test')
|
||||
|
||||
@@ -30,7 +30,7 @@ const resolve = require(`${root}lib/util/resolve`)
|
||||
const { fs } = require(`${root}lib/util/fs`)
|
||||
const glob = require(`${root}lib/util/glob`)
|
||||
const CacheBuster = require(`${root}lib/util/cache_buster`)
|
||||
const Fixtures = require(`${root}test/support/helpers/fixtures`)
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
/**
|
||||
* @type {import('@packages/resolve-dist')}
|
||||
*/
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
require('../spec_helper')
|
||||
|
||||
const plugins = require('../../lib/plugins')
|
||||
const Fixtures = require('../support/helpers/fixtures')
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
|
||||
const pluginsFile = Fixtures.projectPath('plugin-before-browser-launch-deprecation/cypress/plugins/index.js')
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ const config = require(`${root}lib/config`)
|
||||
const { ServerE2E } = require(`${root}lib/server-e2e`)
|
||||
const { SocketE2E } = require(`${root}lib/socket-e2e`)
|
||||
const { SpecsStore } = require(`${root}/lib/specs-store`)
|
||||
const Fixtures = require(`${root}test/support/helpers/fixtures`)
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
const { createRoutes } = require(`${root}lib/routes`)
|
||||
|
||||
const s3StaticHtmlUrl = 'https://s3.amazonaws.com/internal-test-runner-assets.cypress.io/index.html'
|
||||
|
||||
@@ -12,7 +12,7 @@ const { ServerE2E } = require(`${root}lib/server-e2e`)
|
||||
const { SocketE2E } = require(`${root}lib/socket-e2e`)
|
||||
const { SpecsStore } = require(`${root}/lib/specs-store`)
|
||||
const { Automation } = require(`${root}lib/automation`)
|
||||
const Fixtures = require(`${root}/test/support/helpers/fixtures`)
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
const { createRoutes } = require(`${root}lib/routes`)
|
||||
|
||||
const cyPort = 12345
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
const e2e = require('../support/helpers/e2e').default
|
||||
const systemTests = require('@tooling/system-tests/lib/system-tests').default
|
||||
|
||||
// https://github.com/cypress-io/cypress/issues/4313
|
||||
context('cy.visit performance tests', function () {
|
||||
this.retries(3)
|
||||
|
||||
e2e.setup({
|
||||
systemTests.setup({
|
||||
servers: {
|
||||
port: 3434,
|
||||
onServer (app) {
|
||||
@@ -27,7 +27,7 @@ context('cy.visit performance tests', function () {
|
||||
return stdout.replace(/^\d+%\s+of visits to [^\s]+ finished in less than.*$/gm, 'histogram line')
|
||||
}
|
||||
|
||||
e2e.it('passes', {
|
||||
systemTests.it('passes', {
|
||||
onStdout,
|
||||
spec: 'fast_visit_spec.js',
|
||||
snapshot: true,
|
||||
|
||||
@@ -9,7 +9,7 @@ const { expect } = require('chai')
|
||||
const debug = require('debug')('test:proxy-performance')
|
||||
const DebuggingProxy = require('@cypress/debugging-proxy')
|
||||
const HarCapturer = require('chrome-har-capturer')
|
||||
const performance = require('../support/helpers/performance')
|
||||
const performance = require('@tooling/system-tests/lib/performance')
|
||||
const Promise = require('bluebird')
|
||||
const sanitizeFilename = require('sanitize-filename')
|
||||
const { createRoutes } = require(`${root}lib/routes`)
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
it('has expected resolvedNodePath and resolvedNodeVersion', () => {
|
||||
expect(Cypress.config('nodeVersion')).to.eq('system')
|
||||
expect(Cypress.config('resolvedNodePath')).to.eq(Cypress.env('expectedNodePath'))
|
||||
expect(Cypress.config('resolvedNodeVersion')).to.eq(Cypress.env('expectedNodeVersion'))
|
||||
})
|
||||
@@ -1,15 +0,0 @@
|
||||
const fs = require('fs')
|
||||
const zlib = require('zlib')
|
||||
const path = require('path')
|
||||
|
||||
const src = path.join('test/support/fixtures/projects/e2e/static/FiraSans-Regular.woff')
|
||||
const dest = path.join('test/support/fixtures/projects/e2e/static/FiraSans-Regular.woff.gz')
|
||||
|
||||
fs.readFile(src, (err, buf) => {
|
||||
zlib.gzip(buf, (err, zipped) => {
|
||||
console.log(dest)
|
||||
fs.writeFile(dest, zipped, (err, bytes) => {
|
||||
console.log(err, bytes)
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -4,7 +4,7 @@ require(`${root}lib/cwd`)
|
||||
const Promise = require('bluebird')
|
||||
const cache = require(`${root}lib/cache`)
|
||||
const { fs } = require(`${root}lib/util/fs`)
|
||||
const Fixtures = require('../support/helpers/fixtures')
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
|
||||
describe('lib/cache', () => {
|
||||
beforeEach(() => {
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
require('../spec_helper')
|
||||
|
||||
const _ = require('lodash')
|
||||
const path = require('path')
|
||||
const R = require('ramda')
|
||||
const debug = require('debug')('test')
|
||||
const config = require(`${root}lib/config`)
|
||||
@@ -9,6 +8,7 @@ const errors = require(`${root}lib/errors`)
|
||||
const configUtil = require(`${root}lib/util/config`)
|
||||
const findSystemNode = require(`${root}lib/util/find_system_node`)
|
||||
const scaffold = require(`${root}lib/scaffold`)
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
let settings = require(`${root}lib/util/settings`)
|
||||
|
||||
describe('lib/config', () => {
|
||||
@@ -16,6 +16,8 @@ describe('lib/config', () => {
|
||||
this.env = process.env
|
||||
|
||||
process.env = _.omit(process.env, 'CYPRESS_DEBUG')
|
||||
|
||||
Fixtures.scaffold()
|
||||
})
|
||||
|
||||
afterEach(function () {
|
||||
@@ -2037,7 +2039,7 @@ describe('lib/config', () => {
|
||||
})
|
||||
|
||||
it('sets the supportFile to default index.js if it does not exist, support folder does not exist, and supportFile is the default', () => {
|
||||
const projectRoot = path.join(process.cwd(), 'test/support/fixtures/projects/no-scaffolding')
|
||||
const projectRoot = Fixtures.projectPath('no-scaffolding')
|
||||
|
||||
const obj = config.setAbsolutePaths({
|
||||
projectRoot,
|
||||
@@ -2055,7 +2057,7 @@ describe('lib/config', () => {
|
||||
})
|
||||
|
||||
it('sets the supportFile to false if it does not exist, support folder exists, and supportFile is the default', () => {
|
||||
const projectRoot = path.join(process.cwd(), 'test/support/fixtures/projects/empty-folders')
|
||||
const projectRoot = Fixtures.projectPath('empty-folders')
|
||||
|
||||
const obj = config.setAbsolutePaths({
|
||||
projectRoot,
|
||||
@@ -2086,7 +2088,7 @@ describe('lib/config', () => {
|
||||
})
|
||||
|
||||
it('sets the supportFile to index.ts if it exists (without ts require hook)', () => {
|
||||
const projectRoot = path.join(process.cwd(), 'test/support/fixtures/projects/ts-proj')
|
||||
const projectRoot = Fixtures.projectPath('ts-proj')
|
||||
const supportFolder = `${projectRoot}/cypress/support`
|
||||
const supportFilename = `${supportFolder}/index.ts`
|
||||
|
||||
@@ -2113,7 +2115,7 @@ describe('lib/config', () => {
|
||||
})
|
||||
|
||||
it('uses custom TS supportFile if it exists (without ts require hook)', () => {
|
||||
const projectRoot = path.join(process.cwd(), 'test/support/fixtures/projects/ts-proj-custom-names')
|
||||
const projectRoot = Fixtures.projectPath('ts-proj-custom-names')
|
||||
const supportFolder = `${projectRoot}/cypress`
|
||||
const supportFilename = `${supportFolder}/support.ts`
|
||||
|
||||
@@ -2153,7 +2155,7 @@ describe('lib/config', () => {
|
||||
})
|
||||
|
||||
it('sets the pluginsFile to default index.js if does not exist', () => {
|
||||
const projectRoot = path.join(process.cwd(), 'test/support/fixtures/projects/no-scaffolding')
|
||||
const projectRoot = Fixtures.projectPath('no-scaffolding')
|
||||
|
||||
const obj = {
|
||||
projectRoot,
|
||||
@@ -2170,7 +2172,7 @@ describe('lib/config', () => {
|
||||
})
|
||||
|
||||
it('sets the pluginsFile to index.ts if it exists', () => {
|
||||
const projectRoot = path.join(process.cwd(), 'test/support/fixtures/projects/ts-proj-with-module-esnext')
|
||||
const projectRoot = Fixtures.projectPath('ts-proj-with-module-esnext')
|
||||
|
||||
const obj = {
|
||||
projectRoot,
|
||||
@@ -2187,7 +2189,7 @@ describe('lib/config', () => {
|
||||
})
|
||||
|
||||
it('sets the pluginsFile to index.ts if it exists (without ts require hook)', () => {
|
||||
const projectRoot = path.join(process.cwd(), 'test/support/fixtures/projects/ts-proj-with-module-esnext')
|
||||
const projectRoot = Fixtures.projectPath('ts-proj-with-module-esnext')
|
||||
const pluginsFolder = `${projectRoot}/cypress/plugins`
|
||||
const pluginsFilename = `${pluginsFolder}/index.ts`
|
||||
|
||||
@@ -2211,7 +2213,7 @@ describe('lib/config', () => {
|
||||
})
|
||||
|
||||
it('set the pluginsFile to false if it does not exist, plugins folder exists, and pluginsFile is the default', () => {
|
||||
const projectRoot = path.join(process.cwd(), 'test/support/fixtures/projects/empty-folders')
|
||||
const projectRoot = Fixtures.projectPath('empty-folders')
|
||||
|
||||
const obj = config.setAbsolutePaths({
|
||||
projectRoot,
|
||||
@@ -2242,7 +2244,7 @@ describe('lib/config', () => {
|
||||
})
|
||||
|
||||
it('uses custom TS pluginsFile if it exists (without ts require hook)', () => {
|
||||
const projectRoot = path.join(process.cwd(), 'test/support/fixtures/projects/ts-proj-custom-names')
|
||||
const projectRoot = Fixtures.projectPath('ts-proj-custom-names')
|
||||
const pluginsFolder = `${projectRoot}/cypress`
|
||||
const pluginsFile = `${pluginsFolder}/plugins.ts`
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@ require('../spec_helper')
|
||||
|
||||
const config = require(`${root}lib/config`)
|
||||
const files = require(`${root}lib/files`)
|
||||
const FixturesHelper = require(`${root}/test/support/helpers/fixtures`)
|
||||
const FixturesHelper = require('@tooling/system-tests/lib/fixtures')
|
||||
|
||||
describe('lib/files', () => {
|
||||
beforeEach(function () {
|
||||
|
||||
@@ -5,7 +5,7 @@ const Promise = require('bluebird')
|
||||
const config = require(`${root}lib/config`)
|
||||
const fixture = require(`${root}lib/fixture`)
|
||||
const { fs } = require(`${root}lib/util/fs`)
|
||||
const FixturesHelper = require(`${root}/test/support/helpers/fixtures`)
|
||||
const FixturesHelper = require('@tooling/system-tests/lib/fixtures')
|
||||
const os = require('os')
|
||||
const eol = require('eol')
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ const ProjectBase = require(`${root}lib/project-base`).ProjectBase
|
||||
const { openProject } = require('../../lib/open_project')
|
||||
const preprocessor = require(`${root}lib/plugins/preprocessor`)
|
||||
const runEvents = require(`${root}lib/plugins/run_events`)
|
||||
const Fixtures = require('../test/../support/helpers/fixtures')
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
|
||||
const todosPath = Fixtures.projectPath('todos')
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ const task = require(`${root}../../lib/plugins/child/task`)
|
||||
const util = require(`${root}../../lib/plugins/util`)
|
||||
const resolve = require(`${root}../../lib/util/resolve`)
|
||||
const browserUtils = require(`${root}../../lib/browsers/utils`)
|
||||
const Fixtures = require(`${root}../../test/support/helpers/fixtures`)
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
const tsNodeUtil = require(`${root}../../lib/util/ts_node`)
|
||||
|
||||
const runPlugins = require(`${root}../../lib/plugins/child/run_plugins`)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
require('../../spec_helper')
|
||||
|
||||
const Fixtures = require('../../support/helpers/fixtures')
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
const path = require('path')
|
||||
const appData = require(`${root}../lib/util/app_data`)
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ const path = require('path')
|
||||
const commitInfo = require('@cypress/commit-info')
|
||||
const chokidar = require('chokidar')
|
||||
const pkg = require('@packages/root')
|
||||
const Fixtures = require('../support/helpers/fixtures')
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
const { sinon } = require('../spec_helper')
|
||||
const api = require(`${root}lib/api`)
|
||||
const user = require(`${root}lib/user`)
|
||||
|
||||
@@ -3,7 +3,7 @@ import path from 'path'
|
||||
import sinon from 'sinon'
|
||||
import { fs } from '../../lib/util/fs'
|
||||
import { getSpecUrl, checkSupportFile, getDefaultConfigFilePath } from '../../lib/project_utils'
|
||||
import Fixtures from '../support/helpers/fixtures'
|
||||
import Fixtures from '@tooling/system-tests/lib/fixtures'
|
||||
|
||||
const todosPath = Fixtures.projectPath('todos')
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ const ProjectBase = require(`${root}lib/project-base`).ProjectBase
|
||||
const scaffold = require(`${root}lib/scaffold`)
|
||||
const { fs } = require(`${root}lib/util/fs`)
|
||||
const glob = require(`${root}lib/util/glob`)
|
||||
const Fixtures = require(`${root}/test/support/helpers/fixtures`)
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
|
||||
describe('lib/scaffold', () => {
|
||||
beforeEach(() => {
|
||||
|
||||
@@ -6,7 +6,7 @@ const Jimp = require('jimp')
|
||||
const { Buffer } = require('buffer')
|
||||
const dataUriToBuffer = require('data-uri-to-buffer')
|
||||
const sizeOf = require('image-size')
|
||||
const Fixtures = require('../support/helpers/fixtures')
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
const config = require(`${root}lib/config`)
|
||||
const screenshots = require(`${root}lib/screenshots`)
|
||||
const { fs } = require(`${root}lib/util/fs`)
|
||||
|
||||
@@ -15,7 +15,7 @@ const exec = require(`${root}lib/exec`)
|
||||
const preprocessor = require(`${root}lib/plugins/preprocessor`)
|
||||
const { fs } = require(`${root}lib/util/fs`)
|
||||
const open = require(`${root}lib/util/open`)
|
||||
const Fixtures = require(`${root}/test/support/helpers/fixtures`)
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
const firefoxUtil = require(`${root}lib/browsers/firefox-util`).default
|
||||
const { createRoutes } = require(`${root}lib/routes`)
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ const R = require('ramda')
|
||||
const path = require('path')
|
||||
const config = require(`${root}lib/config`)
|
||||
const specsUtil = require(`${root}lib/util/specs`).default
|
||||
const FixturesHelper = require(`${root}/test/support/helpers/fixtures`)
|
||||
const FixturesHelper = require('@tooling/system-tests/lib/fixtures')
|
||||
const debug = require('debug')('test')
|
||||
|
||||
describe('lib/util/specs', () => {
|
||||
|
||||
@@ -5,7 +5,7 @@ import sinon from 'sinon'
|
||||
import snapshot from 'snap-shot-it'
|
||||
import { expect } from 'chai'
|
||||
|
||||
import Fixtures from '../../support/helpers/fixtures'
|
||||
import Fixtures from '@tooling/system-tests/lib/fixtures'
|
||||
import { fs } from '../../../lib/util/fs'
|
||||
import {
|
||||
generateCypressCommand,
|
||||
@@ -19,10 +19,6 @@ import {
|
||||
countStudioUsage,
|
||||
} from '../../../lib/util/spec_writer'
|
||||
|
||||
const mockSpec = Fixtures.get('projects/studio/cypress/integration/unwritten.spec.js')
|
||||
const emptyCommentsSpec = Fixtures.get('projects/studio/cypress/integration/empty-comments.spec.js')
|
||||
const writtenSpec = Fixtures.get('projects/studio/cypress/integration/written.spec.js')
|
||||
|
||||
const exampleTestCommands = [
|
||||
{
|
||||
selector: '.input',
|
||||
@@ -46,10 +42,15 @@ const verifyOutput = (ast) => {
|
||||
}
|
||||
|
||||
describe('lib/util/spec_writer', () => {
|
||||
let readFile
|
||||
let readFile; let mockSpec; let emptyCommentsSpec; let writtenSpec
|
||||
|
||||
// recast doesn't play nicely with mockfs so we do it manually
|
||||
beforeEach(() => {
|
||||
Fixtures.scaffold()
|
||||
mockSpec = fs.readFileSync(Fixtures.projectPath('studio/cypress/integration/unwritten.spec.js'))
|
||||
emptyCommentsSpec = fs.readFileSync(Fixtures.projectPath('studio/cypress/integration/empty-comments.spec.js'))
|
||||
writtenSpec = fs.readFileSync(Fixtures.projectPath('studio/cypress/integration/written.spec.js'))
|
||||
|
||||
readFile = sinon.stub(fs, 'readFile').resolves(mockSpec)
|
||||
sinon.stub(fs, 'writeFile').callsFake((path, output) => {
|
||||
snapshot(output)
|
||||
|
||||
59
patches/snap-shot-core+10.2.0.patch
Normal file
59
patches/snap-shot-core+10.2.0.patch
Normal file
@@ -0,0 +1,59 @@
|
||||
diff --git a/node_modules/snap-shot-core/src/file-system.js b/node_modules/snap-shot-core/src/file-system.js
|
||||
index b2886cd..7d199a0 100644
|
||||
--- a/node_modules/snap-shot-core/src/file-system.js
|
||||
+++ b/node_modules/snap-shot-core/src/file-system.js
|
||||
@@ -21,11 +21,14 @@ const exportObject = require('./utils').exportObject
|
||||
* and we don't want the snapshots to randomly "jump" around and be
|
||||
* saved in an unexpected location.
|
||||
*/
|
||||
-const cwd = process.cwd()
|
||||
+// cache CWD globally so we can reset it in the case where a test
|
||||
+// changes cwd before/after this file is required in a non-deterministic
|
||||
+// fashion (like in the autobalanced system tests)
|
||||
+global.CACHED_CWD_FOR_SNAP_SHOT_IT = process.cwd()
|
||||
/**
|
||||
* Returns a relative path to the original working directory.
|
||||
*/
|
||||
-const fromCurrentFolder = path.relative.bind(null, cwd)
|
||||
+const fromCurrentFolder = (...args) => path.relative(global.CACHED_CWD_FOR_SNAP_SHOT_IT, ...args)
|
||||
const snapshotsFolderName = '__snapshots__'
|
||||
/**
|
||||
* Given relative path, returns same relative path, but inside
|
||||
@@ -34,14 +37,14 @@ const snapshotsFolderName = '__snapshots__'
|
||||
* joinSnapshotsFolder('foo/bar')
|
||||
* // CWD/__snapshots__/foo/bar
|
||||
*/
|
||||
-const joinSnapshotsFolder = path.join.bind(null, cwd, snapshotsFolderName)
|
||||
+const joinSnapshotsFolder = (...args) => path.join(global.CACHED_CWD_FOR_SNAP_SHOT_IT, snapshotsFolderName, ...args)
|
||||
|
||||
// TODO: expose the name of the snapshots folder to the outside world id:16
|
||||
// - <https://github.com/bahmutov/snap-shot-core/issues/245>
|
||||
// Gleb Bahmutov
|
||||
// gleb.bahmutov@gmail.com
|
||||
const snapshotsFolder = fromCurrentFolder(snapshotsFolderName)
|
||||
-debug('process cwd: %s', cwd)
|
||||
+debug('process cwd: %s', global.CACHED_CWD_FOR_SNAP_SHOT_IT)
|
||||
debug('snapshots folder: %s', snapshotsFolder)
|
||||
|
||||
/**
|
||||
@@ -52,7 +55,7 @@ debug('snapshots folder: %s', snapshotsFolder)
|
||||
* we want to form snapshot filenames wrt to the original starting
|
||||
* working directory.
|
||||
*/
|
||||
-const resolveToCwd = path.resolve.bind(null, cwd)
|
||||
+const resolveToCwd = (...args) => path.resolve(global.CACHED_CWD_FOR_SNAP_SHOT_IT, ...args)
|
||||
|
||||
const isSaveOptions = is.schema({
|
||||
sortSnapshots: is.bool
|
||||
diff --git a/node_modules/snap-shot-core/src/index.js b/node_modules/snap-shot-core/src/index.js
|
||||
index b0442f0..fbe55bf 100644
|
||||
--- a/node_modules/snap-shot-core/src/index.js
|
||||
+++ b/node_modules/snap-shot-core/src/index.js
|
||||
@@ -332,6 +332,7 @@ function core (options) {
|
||||
if (expected === undefined) {
|
||||
if (opts.ci) {
|
||||
console.log('current directory', process.cwd())
|
||||
+ console.log('cached cwd', global.CACHED_CWD_FOR_SNAP_SHOT_IT)
|
||||
console.log('new value to save: %j', value)
|
||||
return throwCannotSaveOnCI({
|
||||
value,
|
||||
@@ -1,16 +1,53 @@
|
||||
diff --git a/node_modules/snap-shot-it/src/index.js b/node_modules/snap-shot-it/src/index.js
|
||||
index 0fb68da..154f09d 100644
|
||||
index 0fb68da..d1ff31a 100644
|
||||
--- a/node_modules/snap-shot-it/src/index.js
|
||||
+++ b/node_modules/snap-shot-it/src/index.js
|
||||
@@ -319,6 +319,11 @@ function snapshot (value) {
|
||||
// the code is duplicate from above to get just the key collision error
|
||||
const info = R.assoc('key', e.key, snapshotInfo)
|
||||
info.allowDuplicate = Boolean(snapshotOptions.allowSharedSnapshot)
|
||||
+
|
||||
+ if (info.allowDuplicate) {
|
||||
+ return
|
||||
+ }
|
||||
+
|
||||
debug('current snapshot info %o', info)
|
||||
@@ -16,8 +16,11 @@ const itsName = require('its-name')
|
||||
|
||||
// save current directory right away to avoid any surprises later
|
||||
// when some random tests change it
|
||||
-const cwd = process.cwd()
|
||||
-const relativeToCwd = path.relative.bind(null, cwd)
|
||||
+// cache CWD globally so we can reset it in the case where a test
|
||||
+// changes cwd before/after this file is required in a non-deterministic
|
||||
+// fashion (like in the autobalanced system tests)
|
||||
+global.CACHED_CWD_FOR_SNAP_SHOT_IT = process.cwd()
|
||||
+const relativeToCwd = (...args) => path.relative(global.CACHED_CWD_FOR_SNAP_SHOT_IT, ...args)
|
||||
|
||||
debug('loading snap-shot-it')
|
||||
const EXTENSION = '.js'
|
||||
@@ -239,7 +242,7 @@ function snapshot (value) {
|
||||
compare: R.noop,
|
||||
store: R.noop
|
||||
}
|
||||
- const packageConfigOptions = utils.getPackageConfigOptions(cwd)
|
||||
+ const packageConfigOptions = utils.getPackageConfigOptions(global.CACHED_CWD_FOR_SNAP_SHOT_IT)
|
||||
const opts = utils.mergeConfigOptions(
|
||||
defaultOptions,
|
||||
packageConfigOptions,
|
||||
@@ -254,12 +257,12 @@ function snapshot (value) {
|
||||
debug('prune options %o', pruneSnapshotsOptions)
|
||||
|
||||
const compare = opts.compare
|
||||
- ? utils.load(cwd, opts.compare)
|
||||
+ ? utils.load(global.CACHED_CWD_FOR_SNAP_SHOT_IT, opts.compare)
|
||||
: require('snap-shot-compare')
|
||||
- const store = opts.store ? utils.load(cwd, opts.store) : null
|
||||
+ const store = opts.store ? utils.load(global.CACHED_CWD_FOR_SNAP_SHOT_IT, opts.store) : null
|
||||
|
||||
const preCompare = opts['pre-compare']
|
||||
- ? utils.load(cwd, opts['pre-compare'])
|
||||
+ ? utils.load(global.CACHED_CWD_FOR_SNAP_SHOT_IT, opts['pre-compare'])
|
||||
: R.identity
|
||||
const what = preCompare(value)
|
||||
|
||||
@@ -323,6 +326,9 @@ function snapshot (value) {
|
||||
|
||||
const prevInfo = findExistingSnapshotKey(info)
|
||||
if (prevInfo) {
|
||||
+ if (info.allowDuplicate) {
|
||||
+ return
|
||||
+ }
|
||||
debug('found duplicate snapshot name: %s', prevInfo.key)
|
||||
console.error(
|
||||
'Snapshot error was caused by the duplicate snapshot name'
|
||||
|
||||
@@ -20,7 +20,7 @@ const packages = require('./util/packages')
|
||||
const xvfb = require('../../cli/lib/exec/xvfb')
|
||||
const { transformRequires } = require('./util/transform-requires')
|
||||
const { testStaticAssets } = require('./util/testStaticAssets')
|
||||
const performanceTracking = require('../../packages/server/test/support/helpers/performance.js')
|
||||
const performanceTracking = require('@tooling/system-tests/lib/performance')
|
||||
|
||||
const rootPackage = require('@packages/root')
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ const path = require('path')
|
||||
const Promise = require('bluebird')
|
||||
const os = require('os')
|
||||
const verify = require('../../cli/lib/tasks/verify')
|
||||
const Fixtures = require('../../packages/server/test/support/helpers/fixtures')
|
||||
const Fixtures = require('@tooling/system-tests/lib/fixtures')
|
||||
|
||||
const fs = Promise.promisifyAll(fse)
|
||||
|
||||
|
||||
17
system-tests/.eslintrc.json
Normal file
17
system-tests/.eslintrc.json
Normal file
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"globals": {
|
||||
"expect": "readonly",
|
||||
"mockery": "readonly",
|
||||
"nock": "readonly",
|
||||
"proxyquire": "readonly",
|
||||
"root": "readonly",
|
||||
"sinon": "readonly",
|
||||
"supertest": "readonly"
|
||||
},
|
||||
"extends": [
|
||||
"plugin:@cypress/dev/tests"
|
||||
],
|
||||
"rules": {
|
||||
"no-console": "off"
|
||||
}
|
||||
}
|
||||
70
system-tests/README.md
Normal file
70
system-tests/README.md
Normal file
@@ -0,0 +1,70 @@
|
||||
@tooling/system-tests
|
||||
===
|
||||
|
||||
This package contains Cypress's suite of system tests.
|
||||
|
||||
These tests launch the [Cypress server](../packages/server) process for each test and run different specs and projects under specific environment conditions, to get tests that can be as close to "real world" as possible.
|
||||
|
||||
These tests run in CI in Electron, Chrome, and Firefox under the `system-tests` job family.
|
||||
|
||||
## Running system tests
|
||||
|
||||
```bash
|
||||
yarn test <path/to/test>
|
||||
yarn test test/async_timeouts_spec.js
|
||||
## or
|
||||
yarn test async_timeouts ## shorthand, uses globbing to find spec
|
||||
```
|
||||
|
||||
To keep the browser open after a spec run (for easier debugging and iterating on specs), you can pass the `--no-exit` flag to the test command. Live reloading due to spec changes should also work:
|
||||
|
||||
```sh
|
||||
yarn test test/go_spec.js --browser chrome --no-exit
|
||||
```
|
||||
|
||||
To debug the Cypress process under test, you can pass `--cypress-inspect-brk`:
|
||||
|
||||
```sh
|
||||
yarn test test/go_spec.js --browser chrome --no-exit
|
||||
```
|
||||
|
||||
## Developing tests
|
||||
|
||||
System tests cover the entire Cypress run, so they are good for testing features that do not fit into a normal integration or unit test. However, they do take more resources to run, so consider carefully if you really *need* to write a system test, or if you could achieve 100% coverage via an integration or unit test instead.
|
||||
|
||||
There are two parts to a system test:
|
||||
|
||||
1. A test written using the [`systemTests`](./lib/system-tests) Mocha wrapper that lives in [`./test`](./test), and
|
||||
2. A matching Cypress project that lives in the [`./projects`](./projects) directory.
|
||||
|
||||
For example, if you initialized a new project in `./projects/my-new-project`, and you wanted to assert that 2 tests fail and take a snapshot of the `stdout`, you'd write a test like this:
|
||||
|
||||
```ts
|
||||
// ./test/my-new-project.spec.ts
|
||||
import systemTests from '../lib/system-tests'
|
||||
import Fixtures from '../lib/fixtures'
|
||||
|
||||
describe('my new project', () => {
|
||||
// scaffold projects
|
||||
systemTests.setup()
|
||||
|
||||
systemTests.it('fails as expected', {
|
||||
project: Fixtures.projectPath('my-new-project'),
|
||||
snapshot: true,
|
||||
spec: '*',
|
||||
expectedExitCode: 2
|
||||
})
|
||||
})
|
||||
```
|
||||
|
||||
From here, you could run this test with `yarn test my-new-project`.
|
||||
|
||||
There are many more options available for `systemTests.it` and `systemTests.setup`. You can massage the stdout, do pre-run tasks, set up HTTP/S servers, and more. Explore the typedocs in [`./lib/system-tests`](./lib/system-tests) for more information.
|
||||
|
||||
## Updating snaphots
|
||||
|
||||
Prepend `SNAPSHOT_UPDATE=1` to any test command. See [`snap-shot-it` instructions](https://github.com/bahmutov/snap-shot-it#advanced-use) for more info.
|
||||
|
||||
```bash
|
||||
SNAPSHOT_UPDATE=1 yarn test go_spec
|
||||
```
|
||||
@@ -115,7 +115,7 @@ https://on.cypress.io/cross-origin-script-error
|
||||
│ Pending: 0 │
|
||||
│ Skipped: 0 │
|
||||
│ Screenshots: 5 │
|
||||
│ Video: true │
|
||||
│ Video: false │
|
||||
│ Duration: X seconds │
|
||||
│ Spec Ran: js_error_handling_failing_spec.js │
|
||||
└────────────────────────────────────────────────────────────────────────────────────────────────┘
|
||||
@@ -135,13 +135,6 @@ https://on.cypress.io/cross-origin-script-error
|
||||
rigin script errors -- explains where script errored (failed).png
|
||||
|
||||
|
||||
(Video)
|
||||
|
||||
- Started processing: Compressing to 32 CRF
|
||||
- Finished processing: /XXX/XXX/XXX/cypress/videos/js_error_handling_failing_spec. (X second)
|
||||
js.mp4
|
||||
|
||||
|
||||
====================================================================================================
|
||||
|
||||
(Run Finished)
|
||||
@@ -38,6 +38,8 @@ exports['e2e screenshots / passes'] = `
|
||||
- does not take a screenshot for a pending test
|
||||
✓ adds padding to element screenshot when specified
|
||||
✓ does not add padding to non-element screenshot
|
||||
✓ can pass overwrite option to replace existing filename
|
||||
✓ can set overwrite default option to replace existing filename
|
||||
clipping
|
||||
✓ can clip app screenshots
|
||||
✓ can clip runner screenshots
|
||||
@@ -53,7 +55,7 @@ exports['e2e screenshots / passes'] = `
|
||||
✓ takes another screenshot
|
||||
|
||||
|
||||
20 passing
|
||||
22 passing
|
||||
1 pending
|
||||
6 failing
|
||||
|
||||
@@ -102,8 +104,8 @@ Because this error occurred during a \`after each\` hook we are skipping the rem
|
||||
(Results)
|
||||
|
||||
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Tests: 26 │
|
||||
│ Passing: 20 │
|
||||
│ Tests: 28 │
|
||||
│ Passing: 22 │
|
||||
│ Failing: 5 │
|
||||
│ Pending: 1 │
|
||||
│ Skipped: 0 │
|
||||
@@ -153,6 +155,7 @@ Because this error occurred during a \`after each\` hook we are skipping the rem
|
||||
- /XXX/XXX/XXX/cypress/screenshots/screenshots_spec.js/aut-resize.png (1000x2000)
|
||||
- /XXX/XXX/XXX/cypress/screenshots/screenshots_spec.js/element-padding.png (420x320)
|
||||
- /XXX/XXX/XXX/cypress/screenshots/screenshots_spec.js/non-element-padding.png (600x200)
|
||||
- /XXX/XXX/XXX/cypress/screenshots/screenshots_spec.js/overwrite-test.png (100x50)
|
||||
- /XXX/XXX/XXX/cypress/screenshots/screenshots_spec.js/app-clip.png (100x50)
|
||||
- /XXX/XXX/XXX/cypress/screenshots/screenshots_spec.js/runner-clip.png (120x60)
|
||||
- /XXX/XXX/XXX/cypress/screenshots/screenshots_spec.js/fullPage-clip.png (140x70)
|
||||
@@ -167,10 +170,6 @@ Because this error occurred during a \`after each\` hook we are skipping the rem
|
||||
y long test title aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.png
|
||||
- /XXX/XXX/XXX/cypress/screenshots/screenshots_spec.js/taking screenshots -- reall (1000x660)
|
||||
y long test title aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
||||
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa (1).png
|
||||
|
||||
|
||||
(Video)
|
||||
@@ -186,9 +185,9 @@ Because this error occurred during a \`after each\` hook we are skipping the rem
|
||||
|
||||
Spec Tests Passing Failing Pending Skipped
|
||||
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ ✖ screenshots_spec.js XX:XX 26 20 5 1 - │
|
||||
│ ✖ screenshots_spec.js XX:XX 28 22 5 1 - │
|
||||
└────────────────────────────────────────────────────────────────────────────────────────────────┘
|
||||
✖ 1 of 1 failed (100%) XX:XX 26 20 5 1 -
|
||||
✖ 1 of 1 failed (100%) XX:XX 28 22 5 1 -
|
||||
|
||||
|
||||
`
|
||||
@@ -66,7 +66,7 @@ exports['e2e sessions / session tests'] = `
|
||||
✓ t1
|
||||
✓ t2
|
||||
|
||||
options.validate reruns steps when rejecting
|
||||
options.validate reruns steps when throwing
|
||||
✓ t1
|
||||
✓ t2
|
||||
|
||||
@@ -105,7 +105,17 @@ exports['e2e sessions / session tests'] = `
|
||||
✓ t2
|
||||
|
||||
consoleProps
|
||||
✓ t1
|
||||
- t1
|
||||
|
||||
ignores setting insecure context data when on secure context
|
||||
no cross origin secure origins, nothing to clear
|
||||
✓ sets insecure content
|
||||
✓ nothing to clear - 1/2
|
||||
✓ nothing to clear - 2/2
|
||||
only secure origins cleared
|
||||
✓ sets insecure content
|
||||
✓ switches to secure context - clears only secure context data - 1/2
|
||||
✓ clears only secure context data - 2/2
|
||||
|
||||
errors
|
||||
✓ throws error when experimentalSessionSupport not enabled
|
||||
@@ -113,16 +123,17 @@ exports['e2e sessions / session tests'] = `
|
||||
✓ throws if multiple session calls with same name but different options
|
||||
|
||||
|
||||
49 passing
|
||||
54 passing
|
||||
1 pending
|
||||
|
||||
|
||||
(Results)
|
||||
|
||||
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ Tests: 49 │
|
||||
│ Passing: 49 │
|
||||
│ Tests: 55 │
|
||||
│ Passing: 54 │
|
||||
│ Failing: 0 │
|
||||
│ Pending: 0 │
|
||||
│ Pending: 1 │
|
||||
│ Skipped: 0 │
|
||||
│ Screenshots: 0 │
|
||||
│ Video: false │
|
||||
@@ -138,9 +149,9 @@ exports['e2e sessions / session tests'] = `
|
||||
|
||||
Spec Tests Passing Failing Pending Skipped
|
||||
┌────────────────────────────────────────────────────────────────────────────────────────────────┐
|
||||
│ ✔ session_spec.js XX:XX 49 49 - - - │
|
||||
│ ✔ session_spec.js XX:XX 55 54 - 1 - │
|
||||
└────────────────────────────────────────────────────────────────────────────────────────────────┘
|
||||
✔ All specs passed! XX:XX 49 49 - - -
|
||||
✔ All specs passed! XX:XX 55 54 - 1 -
|
||||
|
||||
|
||||
`
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user