* chore: set up feature/test-burn-in feature branch
* feat: add burnIn Configuration option (currently a no-op) (#27377)
* feat: add the burnIn Configuration to the config package. Option
currently is a no-op
* chore: make burn in experimental
* chore: set experimentalBurnIn to false by default
* feat: add new experimental retries configuration (#27412)
* feat: implement the experimental retries configuration options to pair
with test burn in
* [run ci]
* fix cache invalidation [run ci]
* fix snapshot added in v13 for module api to include test burn in experimentalflag
* chore: fix merge conflict
* chore: add burnInTestAction capability (#27768)
* add burnInTestAction capability
* feat: add burn in capability for cloud
* chore: fix snapshot for record_spec
* feat: implement experimental retries (#27826)
* chore: format the retries/runner snapshot files to make diff easier
* feat: implement experimentalRetries strategies 'detect-flake-and-pass-on-threshold' and 'detect-flake-but-always-fail'. This should not be a breaking change, though it does modify mocha and the test object even when the experiment is not configured. This is to exercise the system and make sure things still work as expected even when we go GA. Test updates will follow in following commits.
* chore: update snapshots from system tests and cy-in-cy tests that now have the cypress test metadata property _cypressTestStatusInfo. tests have been added in the fail-with-[before|after]each specs to visually see the suite being skipped when developing.
* chore: add cy-in-cy tests to verify reporter behavior for pass/fail tests, as well as new mocha snapshots to verify attempts. New tests were needed for this as the 'retries' option in testConfigOverrides currently is and will be invalid for experiment and will function as an override. tests run in the cy-in-cy tests are using globally configured experimentalRetries for the given tested project, which showcases the different behavior between attempts/retries and pass/fail status.
* chore: add unit test like driver test to verify the test object in mocha is decorated/handled properly in calculateTestStatus
* chore: add sanity system tests to verify console reporter output for experimental retries logic. Currently there is a bug in the reporter where the logged status doesnt wait for the aftereach to complete, which impacts the total exitCode and printed status.
* fix: aftereach console output. make sure to fail the test in the appropriate spot in runner.ts and not prematurely, which in turn updates the snapshots for cy-in-cy as the fail event comes later."
* chore: address comments from code review
* fix: make sure hook failures print outer status + attempts when the error is the hook itself.
* chore: improve types within calculateTestStatus inside mocha.ts
* Revert "feat: add burnIn Configuration option (currently a no-op) (#27377)"
This reverts commit c428443079.
* Revert "chore: add burnInTestAction capability (#27768)"
This reverts commit ae3df1a505.
* chore: run snapshot and binary jobs against experimental retries feature branch
* chore: add changelog entry (wip)
* Revert "fix snapshot added in v13 for module api to include test burn in experimentalflag"
This reverts commit bb5046c91e.
* Fix system tests
* Clear CircleCI cache
* Normalize retries config for test execution
* Fixed some unit tests
* update snapshots for newer test metadata
* Fix cy-in-cy snapshots
* update snapshots
* bump cache version
* chore: ensure legacy retry overrides work; reject exp. retries overrides (#28045)
* update changelog
* flip if statement in experimental retries option validation
* refactor invalid experimental retry override for more useful error msg
* revert testConfigOverrides snapshot
* update snapshots for test override sys test
* Update packages/config/src/validation.ts
Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com>
* succinct changelog entry; links to docs for details
* testConfigOverride system test snapshots
* Update .github/workflows/update_v8_snapshot_cache.yml
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
* Update cli/CHANGELOG.md
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
* Update packages/driver/src/cypress.ts
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
* updating cache-version
* improve typescript usage when appending experimental retry options to experiments in Experimenets.vue
* Revert "improve typescript usage when appending experimental retry options to experiments in Experimenets.vue"
This reverts commit b459aba882.
* refactor test config override validation for experimental retry subkeys
* account for error throw differences in browsers in system tests
* bump circle cache
* bump circle cache again
---------
Co-authored-by: astone123 <adams@cypress.io>
Co-authored-by: mabela416 <mabel@cypress.io>
Co-authored-by: Muaz Othman <muaz@cypress.io>
Co-authored-by: Muaz Othman <muazweb@gmail.com>
Co-authored-by: Cacie Prins <cacie@cypress.io>
Co-authored-by: Cacie Prins <cacieprins@users.noreply.github.com>
Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com>
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
Co-authored-by: Matthew Schile <mschile@cypress.io>
* Minor modifications to CLI types so that Cypress and Chai globals are correctly declared not only to be in the global scope but also to be in properties of globalThis, which they are. The change allows composers to explicitly access them through globalThis (e.g., 'globalThis.cy.mount', 'globalThis.expect'). The current typing prohibits this by incorrectly declaring that the objects in question are held in local variables of the global scope rather than in properties of globalThis.
* Reverted a change from previous commit
* fix: Properly type Cypress and Chai globals as properties of the global object
* fix: Properly type Cypress and Chai globals as properties of the global object
* fix: Properly type Cypress and Chai globals as properties of the global object
* update changelog
* fix changelog entry
* fix changelog entry
---------
Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com>
Co-authored-by: Chris Breiding <chrisbreiding@gmail.com>
* chore: update darwin v8 snapshot
* BREAKING CHANGE: set video to false by default (system tests need updating).
* Update cli/CHANGELOG.md
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* chore: update type comments
* chore: update protocol snapshot
* run ci
* run ci
* set video to true for chrome browser crash test
* chore: put in workaround for failing system test spec to be fixed in 27062
* chore: allow retries on actionability tests to be at least one retry as the CI tests run faster without video on
* chore: fix flaky navigation test where done is called multiple times almsot always, but sometimes throws an error
---------
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* feat: Selective CSP header directive stripping from HTTPResponse
- uses `stripCspDirectives` config option
* feat: Selective CSP header directive permission from HTTPResponse
- uses `experimentalCspAllowList` config option
* Address Review Comments:
- Add i18n for `experimentalCspAllowList`
- Remove PR link in changelog
- Fix docs link in changelog
- Remove extra typedef additions
- Update validation error message and snapshot
- Fix middleware negated conditional
* chore: refactor driver test into system tests to get better test
coverage on experimentalCspAllowList options
* Address Review Comments:
- Remove legacyOption for `experimentalCspAllowList`
- Update App desc for `experimentalCspAllowList` to include "Content-Security-Policy-Report-Only"
- Modify CHANGELOG wording
- Specify “never” overrideLevel
- Remove unused validator (+2 squashed commits)
- Add "Addresses" note in CHANGELOG to satisfy automation
- Set `canUpdateDuringTestTime` to `false` to prevent confusion
* chore: Add `frame-src` and `child-src` to conditional CSP directives
* chore: Rename `isSubsetOf` to `isArrayIncludingAny`
* chore: fix CLI linting types
* chore: fix server unit tests
* chore: fix system tests within firefox and webkit
* chore: add form-action test
* chore: update system test snapshots
* chore: skip tests in webkit due to form-action flakiness
* chore: Move 'sandbox' and 'navigate-to' into `unsupportedCSPDirectives`
- Add additional system tests
- Update snapshots and unit test
* chore: update system test snapshots
* chore: fix system tests
* chore: do not run csp tests within firefox or webkit due to flake issues in CI
* chore: attempt to increase intercept delay to avoid race condition
* chore: update new snapshots with video defaults work
* chore: update changelog
---------
Co-authored-by: Bill Glesias <bglesias@gmail.com>
Co-authored-by: Matt Schile <mschile@cypress.io>
* chore: allow users to pass true to videoCompression config and only allow valudes 1-51 inclusively to be passed in
* run ci
* chore: allow zero to be option for CRF as we will coerve it to false and skip compression to have a lossless video, which has the same effect
* Update cli/CHANGELOG.md
* chore: update videoCompression types
* chore: update changelog
* Update cli/CHANGELOG.md
* Update cli/types/cypress.d.ts
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* Update packages/config/src/validation.ts
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* chore: update config snapshots
* Update packages/config/test/validation.spec.ts
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* chore: add system test on videoCompression=true coersion
* chore: document 0 as being false and not a valid CRF option for cypress video compression and make CRF valid values 1-51
* chore: fix types
* chore: fix types
* chore: fix change to log as feature and not bugfix
* chore: fix server side unit tests
---------
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* feat: allow setCookie API to take a hostOnly option
* chore: add jsdoc/typescript description to render to users
* chore: add changelog entry
* [run ci]
* chore: fix types
* chore: fix cookie login tests
* chore: update e2e cookie system tests
* [run ci]
* chore: fix cookie command tests. localhost cookies are calculated as hostOnly, which is consistent with how cypress works today
* chore: fix system tests for cookies.
* [run ci]
* chore: fix system tests
* chore: skip hostOnly assertions in webkit (for now)
* [run ci]
* chore: add property definitions to setCookieOptions
* [run ci]
* chore: add comments to hostOnly prop in firefox when setting a cookie
* feat: set up experimentalUseDefaultDocumentDomain to disallow document.domain overwritting
* use default domain around experimentalUseDefaultDocumentDomain in main iframe and spec bridge iframes. Also adapt CORS policy to use same-origin if experimental flag is set
* run ci
* fix: add insertion of experimental flag where is was needed/missing
* chore: add system test to exercise experimental flag for expected behavior
* fix: fix issues with template updates to conform to squirrelly v7
* fix: update config tests to include new experimental flag
* run ci
* fix: trailing whitespace [run ci]
* chore: update snapshot
* run ci
* fix: update proxy unit tests to account for experimentalUseDefaultDocumentDomain
* run ci
* fix: Allow component tests with special characters in filepath (#25299)
feat: cut over experimental flag to take list of known problematic domains via string/glob pattern
run ci
chore: update system test and fix broken config
* fix: fix server unit and integration tests. integration tests should no longer use google to test against injection as we do not inject document.domain on google domains
* run ci
* run ci
* fix: server integration tests where google documents are expected to receive document.domain injection. Kept test same by changing URL
* run ci
* fix: update server test with mssing unupdated assertions
* run ci
* fix: turn off experimental flag by default while recommending sane defaults to users to configure
* run ci
* chore: fix typings [run ci]
* run ci
* chore: make experiment an e2e option only
* run ci
* chore: address comments in code review
* chore: rename experimentalUseDefaultDocumentDomain to experimentalSkipDomainInjection
* fix regression in shouldInjectionDocumentDomain utility function and add unit tests
* run ci
* chore: rename documentSuperDomainIfExists to superDomain [run ci]
* chore: address comments from code review
* chore: just pass opts through to policyForDomain
* run ci
Co-authored-by: Mike Plummer <mike-plummer@users.noreply.github.com>
* fix: Improve TypeScript support for custom queries
* Typo fix
* Fix TS more
* Move 'ensures' off of cy and onto Cypress
* Type fixes
* One missed replacement
* Test fixes
* Properly pass in to ensureRunnable
* Add .eslintignores to quiet down warnings
* Review feedback
* Update cli/types/cypress.d.ts
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
