485 Commits

Author SHA1 Message Date
Zach Bloomquist f2100a8bba test(system-tests): support npm for test projects (#20664) 2022-03-22 13:28:21 +00:00
Zach Bloomquist 3c286177cc fix: use unique install cache folders for betas (#20296) 2022-03-04 22:22:39 +00:00
Emily Rohrbough ef3be70944 chore: update release scripts to work with AWS SSO (#20438) 2022-03-04 12:31:27 -06:00
Emily Rohrbough 469dfe4285 chore: speed up mac build times in CI (#20473) 2022-03-04 11:25:44 -06:00
Zach Bloomquist ffd3627e43 test: migrate module_api to system tests (#20265)
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
2022-02-25 23:12:05 +00:00
Zach Bloomquist 95700aed46 chore: fix "cannot find module" in clone-repo-and-checkout-release-branch (#20293) 2022-02-25 15:58:49 +00:00
Zach Bloomquist 27e3dc3dc5 chore: fix CI cache state for darwin (#20339) 2022-02-24 09:07:44 -05:00
Zach Bloomquist 1fb16b0c56 chore: fix cache keys to include PLATFORM (#20279)
* chore: fix cache keys to include PLATFORM

* build this branch

* try using platform_key
2022-02-20 16:22:08 -05:00
Zach Bloomquist ad2f4de844 test(system-tests): support docker-based tests against built binary (#20250)
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
2022-02-18 23:37:22 +00:00
Zach Bloomquist 42b0fce6dc test: replace cypress-test-example-repos coverage + remove bump (#20186) 2022-02-16 18:53:04 +00:00
BlueWinds ac1faf4965 Merge remote-tracking branch 'origin/develop' into issue-19403-perf-reporter-changes 2022-02-14 08:19:17 -08:00
Zach Bloomquist a1e6d5cad1 test: remove redundant "other projects" CI jobs (#20133)
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
2022-02-11 18:18:53 +00:00
Brian Mann 29841f32b9 feat: redesign server errors (#20072)
* chore: rename errors.js -> errors.ts

* refactor: type safety on errors

* refactor: add err_template for consistent error formatting

* fix a few system tests

* fix tests; update snapshots

* Fix types

* normalize snapshot - remove chalk ansi colors

* more unit test fixes

* more system test fixes

* circleci build

* backtick always in stdout, fix error formatting and failing snapshots

* refactor: create @packages/errors

* fix import

* fix import

* fixing build / tests

* remove extraneous file

* move warnIfExplicitCiBuildId

* fix build / tests

* Fix

* error, type fixes, documentation, standardize child process error serialization

* fix import

* build errors on install

* wrote specs generating visual images of all errors

* remove unused dep

* sanitize stack traces

* add image diffing

- if base images don't exist, create them
- if base images don't match and local, overwrite them, if in CI throw
- if base images are stale and local, delete them, if in CI throw

* remove Courier New + MesloLGS NF font

* type fixes, remove Bluebird, general cleanup

* TS Cleanup

* skip typecheck on tests for now

* yarn.lock

* fix @types/chai version

* fix yarn.lock

* Different version of mocha types so it isnt patched

* errors spec snapshot

* CI fix

* fixes

* store snapshot images in circle artifacts

* dont change artifact destination prefix

* use Courier Prime

* antialias the text

* decrease pixelmatch threshold, fail in CI only when changed pixels > 100

* increase timeout

* overflow: hidden, remove new Promise, add debug logging

Co-Authored-By: Tim Griesser <tgriesser@gmail.com>

* run unit tests w/ concurrency=1

* unique window per file

* disable app hardware acceleration + use in process gpu + single process

* do not do image diffing

- conditionally convert html to images
- store html snapshots
- do not store images in git

* store snapshot html

* Merge branch 'tgriesser/chore/refactor-errors' of https://github.com/cypress-io/cypress into tgriesser/chore/refactor-errors

* remove concurrency

* fix assertion

* fixing ci

* Link in readme

* pass the browsers to listItems

* fix: build @packages/errors in CI, defer import to prevent errors locally

* Merge branch 'develop' into tgriesser/chore/refactor-errors

* develop:
  chore: fix cypress npm package artifact upload path (#20023)
  chore(driver): move cy.within logic into it's own file (#20036)
  chore: update automerge workflows (#19982)
  fix(selectFile): use target window's File/DataTransfer classes (#20003)
  chore: Update Chrome (stable) to 98.0.4758.80 and Chrome (beta) to 98.0.4758.80 (#19995)
  fix: Adjust ffmpeg CLI args for performance (#19983)
  build: allow unified to run cypress on Apple Silicon (arm64) (backport #19067 to 9.x) (#19968)

* fix run-if-ci.sh

* remove dead code

* Mark the .html files as generated in gitattributes

* fix running single error case, slice out more of the brittle stack

* remove additional brittle stack line

* firefox web security error

* nest inside of describe

* reformat and redesign errors

* more error cleanup and standardization

* additional formatting of errors, code cleanup, refactoring

* update ansi colors to match terminal colors

* cleanup remaining loose ends, update several errors, compact excess formatters

* fix types

* additional formatting, remove TODO's, ensure no [object Object] in output

* add test for 412 server response on invalid schema

* update unknown dashboard error on creating run

* use fs.access instead of fs.stat for perf

* added PLUGINS_FILE_NOT_FOUND error

- separated out from PLUGINS_FILE_ERROR
- add system tests for both cases
- update snapshots
- remove stack trace from PLUGINS_FILE_NOT_FOUND fka PLUGINS_FILE_ERROR

* add plugins system test around plugins synchronously throwing on require

* remove forcing process.cwd() to be packages/server, update affected code

- this was a long needed hangover from very old code that was doing unnecessary things due to respawning electron from node and handling various entrypoints into booting cypress
- this also makes the root yarn dev and dev-debug work correctly because options.cwd is set correctly now

* perf: lazy load chalk

* remove excessive line since the file exists but is invalid

* fix types

* add system test when plugins function throws a synchronous error

* create new PLUGINS_INVALID_EVENT_ERROR, wire it up correctly for error template use

- properly pass error instance from child to ensure proper user stack frames
- move error display code into the right place

* only show a single stack trace, either originalError or internal cypressError

* push error html snapshots

* fix tests, types

* fix test

* fix failing tests

* fix tests

* fixes lots of broken tests

* more test fixes

* fixes more tests

* fix type checking

* wip: consistent handling of interpolated values

* feat: fixing up errors, added simple error comparison tool

* wrapping up error formatting

* Fixes for unit tests

* fix PLUGINS_VALIDATION_ERROR

* fix fs.readdir bug, show rows even if there's only markdown formatting [SKIP CI]

* when in base-list, show full width of errors

* Fix type errors

* added searching and filtering for files based on error name

* fix: system tests

* updated NO_SPECS_FOUND error to properly join searched folder + pattern

- join patterns off of process.cwd, not projectRoot
- highlight original specPattern in yellow, baseDir in blue
- add tests

* fixes failing tests

* fix test

* preserve original spec pattern, display relative to projectRoot for terminal banner

* make the nodeVersion path display in gray, not white

* fix tests, pass right variables

* fix chrome:canary snapshots

* update snapshots

* update snapshot

* strip newlines caused by "Still waiting to connect to {browser}..."

* don't remove the snapshotHtmlFolder in CI, add additional verification snapshots match to error keys symmetrically

* update snapshot

* update snapshot

* update snapshot

* update snapshot

* update snapshot

* update snapshot

* update gitignore

* fix snapshot

* update snapshot html

* update logic for parsing the resolve pattern matching, add tests

* update snapshots

* update snapshot

* update snapshot

* update snapshot

* fix failing test

* fix: error_message_spec

* fix snapshot

* run each variant through an it(...) so multiple failures are received

* add newlines to multiline formatters, add fmt.stringify, allow format overrides

* stringify invalid return values from plugins

* move config validation errors into packages/errors, properly highlight and stringify values

* add component testing yarn commands

* fix the arrow not showing on details

* fix typescript error

* fixed lots of poorly written tests that weren't actually testing anything. created settings validation error when given a string validation result.

* fixes tests

* fixes tests, adds new error template for validating within an array list (for browser validation)

* remove dupe line

* fix copy for consistency, update snapshots

* remove redundant errors, standardize formatting and phrasing

* more formatting

* remove excess snapshots

* prune out excessive error snapshot html files when not in CI

* add missing tests, add case for when config validation fails without a fileType

* fixes test

* update snapshot

* update snapshot

* update snapshot

* sort uniqErrors + errorKeys prior to assertion - assert one by one

* add system test for binding to an event with the wrong handler

* fixes tests

* set more descriptive errors when setting invalid plugin events, or during plugin event validation

* remove duplicate PLUGINS_EVENT_ERROR, collapse into existing error

* use the same multiline formatting as @packages/errors

* standardize verbiage and highlighting for consistency

* fix incorrect error path

* fixes tests, standardized and condensed more language

* Update packages/errors/src/errors.ts

Co-authored-by: Ryan Manuel <ryanm@cypress.io>

* Update guides/error-handling.md

Co-authored-by: Ryan Manuel <ryanm@cypress.io>

* Update guides/error-handling.md

Co-authored-by: Ryan Manuel <ryanm@cypress.io>

* added some final todo's

* fix types

Co-authored-by: Tim Griesser <tgriesser10@gmail.com>
Co-authored-by: Tim Griesser <tgriesser@gmail.com>
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
2022-02-11 02:06:07 -05:00
BlueWinds 961e764037 Move sending root event to own script 2022-02-10 14:21:44 -08:00
Zach Bloomquist be4c78b799 test: fix awesome-typescript-loader test and remove test-binary job (#20131) 2022-02-10 18:05:44 +00:00
BlueWinds fa0b68af52 Fix path 2022-02-08 14:50:04 -08:00
BlueWinds 57decf45ef Include honeycomb key so we can send root event 2022-02-08 13:21:57 -08:00
BlueWinds 9d433c764d Send root honeycomb event even if node_modules cache already exists 2022-02-08 12:50:24 -08:00
BlueWinds 705262fb76 Fix 2022-02-08 10:48:17 -08:00
BlueWinds b9c0d5e59b fix indentation 2022-02-08 10:37:52 -08:00
BlueWinds f694bd2bb6 Another attempt to wrap all events into one trace 2022-02-08 10:33:15 -08:00
Emily Rohrbough 17f272e9ee chore: fix cypress npm package artifact upload path (#20023) 2022-02-04 12:00:08 -06:00
Emily Rohrbough 99f24863a2 fix: move node 17 Check from Binary to CLI (#19977) 2022-01-31 12:03:36 -06:00
Blue F f5cb2b9a7b test: Honeycomb system-test reporter (#19855)
* Mocha performance reporter

Co-authored-by: Tim Griesser <tgriesser10@gmail.com>
2022-01-27 09:53:45 -08:00
Mark Noonan d055fdc142 chore: use commit sha for percy nonce to unite reruns (#19765)
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
2022-01-21 09:50:38 -05:00
Lachlan Miller 1305cca9f1 fix: rename specs to correctly match convention (#19641)
* fix: rename specs to correctly match convention

* Remove underscore from TESTFILES glob pattern

Co-authored-by: Zach Bloomquist <github@chary.us>
2022-01-12 07:10:14 +10:00
Emily Rohrbough aea8b2b788 chore: trigger downstream project tests for windows workflow (#19381) 2021-12-17 11:01:20 -06:00
Tim Griesser 8262f80d1f fix: patch-package is not applied in dist'ed build (#19239) 2021-12-03 18:45:34 -05:00
Emily Rohrbough 4d35779c8a chore: Clean up CircleCI Configuraiton (#19212) 2021-12-03 08:55:10 -06:00
Emily Rohrbough 6b15468579 chore: Migrate Windows Tests from Appveyor to CircleCI (#19037)
Co-authored-by: Zach Bloomquist <git@chary.us>
Co-authored-by: ElevateBart <ledouxb@gmail.com>
2021-12-02 14:20:06 -06:00
renovate[bot] 9bf8bf2212 fix(deps): update dependency @cypress/request to v2.88.10 🌟 (#19099)
Co-authored-by: Renovate Bot <bot@renovateapp.com>
Co-authored-by: Chris Breiding <chrisbreiding@gmail.com>
Co-authored-by: Zach Bloomquist <git@chary.us>
2021-11-29 13:59:18 -05:00
Zach Bloomquist 71d92e0e49 test: node_modules installs for system-tests, other improvements (#18574)
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
2021-11-24 20:10:28 +00:00
Emily Rohrbough 5e2de0c9ce chore: remove flaky ci jobs for main builds (#19071) 2021-11-23 14:32:58 -06:00
Tim Griesser e4f5b106d4 feat: use hoisted yarn install in binary build (#17285) 2021-11-20 14:19:51 -05:00
Chris Breiding 4638668785 have conduit app wait on localhost:3000 2021-11-10 13:35:15 -05:00
Zach Bloomquist b338d91eb8 fix install-required-node 2021-11-10 11:50:02 -05:00
Chris Breiding 8f93f179d4 use --legacy-peer-deps 2021-11-10 11:42:54 -05:00
Zach Bloomquist 68c95b03cd fix darwin node install 2021-11-09 17:31:05 -05:00
renovate[bot] 3095d733e9 feat(deps): update dependency electron to v15 🌟 (#18317)
Co-authored-by: Renovate Bot <bot@renovateapp.com>
Co-authored-by: Zach Bloomquist <git@chary.us>
Co-authored-by: David Munechika <davidam1203@gmail.com>
2021-11-09 20:52:17 +00:00
Emily Rohrbough 99c8d466c8 feat: create config package for config validation (#18589) 2021-11-09 13:19:05 -06:00
Jennifer Shehane e8ab12dc03 run develop tests against 9.0 branch 2021-11-02 08:47:53 -06:00
Jennifer Shehane 0273f3ebae Merge branch 'develop' into 9.0-release 2021-11-01 11:58:55 -06:00
Jennifer Shehane 4cb97b3472 chore: Increase paralleled machines for desktop-gui tests (#18725) 2021-11-01 09:45:11 -06:00
Emily Rohrbough e396956534 feat: remove windows 32-bit support (#18630)
Co-authored-by: Jennifer Shehane <jennifer@cypress.io>
Co-authored-by: Zach Bloomquist <git@chary.us>
2021-10-29 08:54:02 -05:00
Jessica Sachs a045e4f59a chore: move server e2e tests to system-tests (#16354)
Co-authored-by: Brian Mann <brian.mann86@gmail.com>
Co-authored-by: Zach Bloomquist <git@chary.us>
Co-authored-by: Zach Bloomquist <github@chary.us>
2021-10-18 19:53:14 +00:00
Zach Bloomquist b3aae8e1ca chore: use circleci timings split for e2e tests (#18367) 2021-10-07 15:25:51 +00:00
Tim Griesser e1a3fa2646 chore: fix semantic-release dry run (#18332) 2021-10-01 14:37:06 -04:00
Ben Kucera 9f9cf9e8b6 chore: fix broken firefox focus tests, bump resource class of chrome-beta job (#18133) 2021-09-21 09:34:48 -04:00
Tim Griesser 863e3680bb fix: GH env variable for test other projects (#18147)
* use GH_TOKEN rather than creds var
2021-09-17 15:39:12 -04:00
Tim Griesser 3f39d7bf84 chore: use dedicated test-runner:npm-release scope in circle (#18112) 2021-09-15 14:27:43 -04:00