* chore: set up feature/test-burn-in feature branch
* feat: add burnIn Configuration option (currently a no-op) (#27377)
* feat: add the burnIn Configuration to the config package. Option
currently is a no-op
* chore: make burn in experimental
* chore: set experimentalBurnIn to false by default
* feat: add new experimental retries configuration (#27412)
* feat: implement the experimental retries configuration options to pair
with test burn in
* [run ci]
* fix cache invalidation [run ci]
* fix snapshot added in v13 for module api to include test burn in experimentalflag
* chore: fix merge conflict
* chore: add burnInTestAction capability (#27768)
* add burnInTestAction capability
* feat: add burn in capability for cloud
* chore: fix snapshot for record_spec
* feat: implement experimental retries (#27826)
* chore: format the retries/runner snapshot files to make diff easier
* feat: implement experimentalRetries strategies 'detect-flake-and-pass-on-threshold' and 'detect-flake-but-always-fail'. This should not be a breaking change, though it does modify mocha and the test object even when the experiment is not configured. This is to exercise the system and make sure things still work as expected even when we go GA. Test updates will follow in following commits.
* chore: update snapshots from system tests and cy-in-cy tests that now have the cypress test metadata property _cypressTestStatusInfo. tests have been added in the fail-with-[before|after]each specs to visually see the suite being skipped when developing.
* chore: add cy-in-cy tests to verify reporter behavior for pass/fail tests, as well as new mocha snapshots to verify attempts. New tests were needed for this as the 'retries' option in testConfigOverrides currently is and will be invalid for experiment and will function as an override. tests run in the cy-in-cy tests are using globally configured experimentalRetries for the given tested project, which showcases the different behavior between attempts/retries and pass/fail status.
* chore: add unit test like driver test to verify the test object in mocha is decorated/handled properly in calculateTestStatus
* chore: add sanity system tests to verify console reporter output for experimental retries logic. Currently there is a bug in the reporter where the logged status doesnt wait for the aftereach to complete, which impacts the total exitCode and printed status.
* fix: aftereach console output. make sure to fail the test in the appropriate spot in runner.ts and not prematurely, which in turn updates the snapshots for cy-in-cy as the fail event comes later."
* chore: address comments from code review
* fix: make sure hook failures print outer status + attempts when the error is the hook itself.
* chore: improve types within calculateTestStatus inside mocha.ts
* Revert "feat: add burnIn Configuration option (currently a no-op) (#27377)"
This reverts commit c428443079.
* Revert "chore: add burnInTestAction capability (#27768)"
This reverts commit ae3df1a505.
* chore: run snapshot and binary jobs against experimental retries feature branch
* chore: add changelog entry (wip)
* Revert "fix snapshot added in v13 for module api to include test burn in experimentalflag"
This reverts commit bb5046c91e.
* Fix system tests
* Clear CircleCI cache
* Normalize retries config for test execution
* Fixed some unit tests
* update snapshots for newer test metadata
* Fix cy-in-cy snapshots
* update snapshots
* bump cache version
* chore: ensure legacy retry overrides work; reject exp. retries overrides (#28045)
* update changelog
* flip if statement in experimental retries option validation
* refactor invalid experimental retry override for more useful error msg
* revert testConfigOverrides snapshot
* update snapshots for test override sys test
* Update packages/config/src/validation.ts
Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com>
* succinct changelog entry; links to docs for details
* testConfigOverride system test snapshots
* Update .github/workflows/update_v8_snapshot_cache.yml
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
* Update cli/CHANGELOG.md
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
* Update packages/driver/src/cypress.ts
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
* updating cache-version
* improve typescript usage when appending experimental retry options to experiments in Experimenets.vue
* Revert "improve typescript usage when appending experimental retry options to experiments in Experimenets.vue"
This reverts commit b459aba882.
* refactor test config override validation for experimental retry subkeys
* account for error throw differences in browsers in system tests
* bump circle cache
* bump circle cache again
---------
Co-authored-by: astone123 <adams@cypress.io>
Co-authored-by: mabela416 <mabel@cypress.io>
Co-authored-by: Muaz Othman <muaz@cypress.io>
Co-authored-by: Muaz Othman <muazweb@gmail.com>
Co-authored-by: Cacie Prins <cacie@cypress.io>
Co-authored-by: Cacie Prins <cacieprins@users.noreply.github.com>
Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com>
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
Co-authored-by: Matthew Schile <mschile@cypress.io>
* feat: update to electron 25 and bump node dependencies. need to remove
custom docker image
* chore: update node versions to 18+ [run ci]
* chore: bump @types/node from v16 to v18 and bump the typescript supported CLI version from 3.9 to 4.4
* update mock-fs to 5.2.0 to fix BigInt issue (see https://github.com/tschaub/mock-fs/releases/tag/v5.1.4) [run ci]
* chore: update electron integrity check to pass create binary job
* chore: fix issues with achrinza/node-ipc not yet having node 18 engine support (only up to 17 by patching from 9.2.2 to 9.2.5 as seen in https://github.com/achrinza/node-ipc/pull/38. vue/cli-service is not maintained and we should migrate these over to vue create at some point in the near future to get rid of these resolutions
* allow for TLSv1 tests to work with node 18 as the express server running node 18 with tlsv1 needs to allow legacy ciphers
* chore: update snyk workflows to use node 18
* chore: add changelog
* update timeout
* more cleanup during binary build
* bump cache and run ci. include ignore engines for rwa and get rid of
16.16 image references
* chore: update FF tests to 115 as serialization now works with errors and click events do not get fired on buttons being typed into
* chore: don't error when symlink already exists
* chore: fix serialization test for newer versions of firefox
* chore: fix CI config
* chore: fix chrome system tests to work with chrome 114. updates mostly caused by bugs in screen height in chrome. see https://bugs.chromium.org/p/chromium/issues/detail?id=1416398
* chore: test binary against vite update in RWA
* remove wait for RWA as it shouldnt be needed for vite. link example recipes update to node 18
chore: remove commented out code
* chore: print message when DISABLE_SNAPSHOT_REQUIRE is set
* chore: clean out unneeded dependencies always
* chore: remove trailing space
* fix: propagate click events for enter and typing on firefox 106 or later
* chore: fix changelog failures
* fix: correctly simulate click events for buttons on keyup and space type in Firefox versions greater than 91 and simulate click for buttons on enter in Firefox versions greater than or equal to 106
* chore: add documentation to type to clarify firefox synthetic events
* chore: update protocol snapshot as order or log messages seems to have changed
* update comments
* sort commandLogChanged events for protocol
* chore: remove PR ids from CRA and CER as the PRs are merged into develop
---------
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
Co-authored-by: Chris Breiding <chrisbreiding@gmail.com>
Co-authored-by: Matt Schile <mschile@cypress.io>
* chore: disable video compression
* system test updates
* moar system test fixes
* a couple of telemetry tweaks
* more intelligent defaults, don't touch firefox.
* trying this
* probably finally fix firefox system tests
* unformat plz
* ugh, also add a test
* plz stop formatting that file
* re-enable firefox tests
* Change log
* don't save videos to artifacts
* this will be fixed in another pr
* quoth the raven, quotes matter don't mess them up
* fix(angular): mount cy-root in original location
* correctly fix insertRootElement to preserves html hierarchy, trim system-tests
---------
Co-authored-by: Zachary Williams <ZachJW34@gmail.com>
* testing: try disabling uTimesSync and see what happens
* build binaries [run ci]
* fix: touch component index file instead of browser.js
* build binaries [run ci]
* update test
* fix test
* add test for custom HTML file in config
* use existing component index in webpack-dev-server unit tests
---------
Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com>
* fix: remove server and route commands
* This won't totally pass but it's a step in the right direction.
* Should be down to only two failing tests due to abort.
* fix the alias.<thing> api for intercept to be equivalent to route.
* adjust timeouts to reduce flake, ideally
* skip xhr abort tests
* More clean up
* Increase timeout
* trick next version
* fix unit tests probably
* yeah sure
* Apply suggestions from code review
Co-authored-by: Blue F <blue@cypress.io>
* remove addXhrLog
* Remove more usage of XHR stemming from proxy-logging
* Remove a couple of files that found their way back with the merge from release/12.0.0
* remove more server types
* increase timeouts again to prevent flake
Co-authored-by: Blue F <blue@cypress.io>
Co-authored-by: Zach Bloomquist <git@chary.us>
* fix: remove mountHook function for React
* fix: remove `unmount` from @cypress/react exports
* fix: remove `unmount` from the @cypress/react readme
* fix: don't alias React components on mount
* fix: remove `mountCallback` from Vue adapters
* fix: remove style injection utility functions from `mount-utils`
* fix: fix React tests
* fix: fix more React tests
* fix: fix more tests
* fix: fix screenshot test styles
* fix: update documentation around mount-utils styles; fix tests
* fix: update Vue docs to use `props` key rather than `propsData`
* fix: add test styles back in
* update unmount test and export getContainerEl for back compat
* better errors
* docs
* error for unmount
* test for error
* fix last test
* adjust language to reflect removed methods
* one last deprecation
* fix error
* wip - [skip ci]
* use proxy to catch errors
* deprecate alias
* update tests
* update on link
* use on links properly
* revert changes
Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com>
* fix: allow to override global services in Angular component tests
* move DI tests into their own context
* exclude `componentProviders` when a component is mounted with a template
* - revert and remove `componentProviders`
- run `eslint --fix` on `mount.cy.ts`
- add tests that use `TestBed.inject`
Co-authored-by: Zachary Williams <ZachJW34@gmail.com>
* fix(react18): unmount component with react18 API
* Add null check and explicit typing per code review
* update snapshots and make code more concise
Co-authored-by: Zachary Williams <zachjw34@gmail.com>
Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com>
