* chore: set up feature/test-burn-in feature branch
* feat: add burnIn Configuration option (currently a no-op) (#27377)
* feat: add the burnIn Configuration to the config package. Option
currently is a no-op
* chore: make burn in experimental
* chore: set experimentalBurnIn to false by default
* feat: add new experimental retries configuration (#27412)
* feat: implement the experimental retries configuration options to pair
with test burn in
* [run ci]
* fix cache invalidation [run ci]
* fix snapshot added in v13 for module api to include test burn in experimentalflag
* chore: fix merge conflict
* chore: add burnInTestAction capability (#27768)
* add burnInTestAction capability
* feat: add burn in capability for cloud
* chore: fix snapshot for record_spec
* feat: implement experimental retries (#27826)
* chore: format the retries/runner snapshot files to make diff easier
* feat: implement experimentalRetries strategies 'detect-flake-and-pass-on-threshold' and 'detect-flake-but-always-fail'. This should not be a breaking change, though it does modify mocha and the test object even when the experiment is not configured. This is to exercise the system and make sure things still work as expected even when we go GA. Test updates will follow in following commits.
* chore: update snapshots from system tests and cy-in-cy tests that now have the cypress test metadata property _cypressTestStatusInfo. tests have been added in the fail-with-[before|after]each specs to visually see the suite being skipped when developing.
* chore: add cy-in-cy tests to verify reporter behavior for pass/fail tests, as well as new mocha snapshots to verify attempts. New tests were needed for this as the 'retries' option in testConfigOverrides currently is and will be invalid for experiment and will function as an override. tests run in the cy-in-cy tests are using globally configured experimentalRetries for the given tested project, which showcases the different behavior between attempts/retries and pass/fail status.
* chore: add unit test like driver test to verify the test object in mocha is decorated/handled properly in calculateTestStatus
* chore: add sanity system tests to verify console reporter output for experimental retries logic. Currently there is a bug in the reporter where the logged status doesnt wait for the aftereach to complete, which impacts the total exitCode and printed status.
* fix: aftereach console output. make sure to fail the test in the appropriate spot in runner.ts and not prematurely, which in turn updates the snapshots for cy-in-cy as the fail event comes later."
* chore: address comments from code review
* fix: make sure hook failures print outer status + attempts when the error is the hook itself.
* chore: improve types within calculateTestStatus inside mocha.ts
* Revert "feat: add burnIn Configuration option (currently a no-op) (#27377)"
This reverts commit c428443079.
* Revert "chore: add burnInTestAction capability (#27768)"
This reverts commit ae3df1a505.
* chore: run snapshot and binary jobs against experimental retries feature branch
* chore: add changelog entry (wip)
* Revert "fix snapshot added in v13 for module api to include test burn in experimentalflag"
This reverts commit bb5046c91e.
* Fix system tests
* Clear CircleCI cache
* Normalize retries config for test execution
* Fixed some unit tests
* update snapshots for newer test metadata
* Fix cy-in-cy snapshots
* update snapshots
* bump cache version
* chore: ensure legacy retry overrides work; reject exp. retries overrides (#28045)
* update changelog
* flip if statement in experimental retries option validation
* refactor invalid experimental retry override for more useful error msg
* revert testConfigOverrides snapshot
* update snapshots for test override sys test
* Update packages/config/src/validation.ts
Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com>
* succinct changelog entry; links to docs for details
* testConfigOverride system test snapshots
* Update .github/workflows/update_v8_snapshot_cache.yml
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
* Update cli/CHANGELOG.md
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
* Update packages/driver/src/cypress.ts
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
* updating cache-version
* improve typescript usage when appending experimental retry options to experiments in Experimenets.vue
* Revert "improve typescript usage when appending experimental retry options to experiments in Experimenets.vue"
This reverts commit b459aba882.
* refactor test config override validation for experimental retry subkeys
* account for error throw differences in browsers in system tests
* bump circle cache
* bump circle cache again
---------
Co-authored-by: astone123 <adams@cypress.io>
Co-authored-by: mabela416 <mabel@cypress.io>
Co-authored-by: Muaz Othman <muaz@cypress.io>
Co-authored-by: Muaz Othman <muazweb@gmail.com>
Co-authored-by: Cacie Prins <cacie@cypress.io>
Co-authored-by: Cacie Prins <cacieprins@users.noreply.github.com>
Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com>
Co-authored-by: Ryan Manuel <ryanm@cypress.io>
Co-authored-by: Matthew Schile <mschile@cypress.io>
* chore: update ts-loader from v8 to v9
* chore: update sass-loader from v10 to v13
* chore: update postcss-loader from v4 to v7
* chore: update mini-css-extract-plugin from v1 to v2
* chore: update html-webpack-plugin from v4 to v5
* chore: update css-loader from v5 to v6
* chore: update copy-webpack-plugin from v6 to v 11
* chore: update babel and babel loader to latest
* chore: remove Url import from validation as it should be available globally
* chore: update coffee-loader to latest and coffeescript to 2.6.0. coffee
script > 2.6.0 has issues with packheard transpilation, which likely
isnt worth the squeeze here to update it further
* chore: fix react component tests with webpack 4 as babl loader 9 only works with webpack 5, so we need ot pin the dep to 8 in the system-test project
* chore: remove dependency on file-loader and opt for ubilt in assert handling where applicable. Additionally, remove react-svg-loader and move to svgr loader inside the runner to bundle svgs needed for react components. Make sure that all SVGs are being imported correctly
chore: tell webpack preprocessor to process svgs inside of e2e tests for
the reporter. We did not need this before because we were using
react-svg-loader, which was used directly in source to process the svg
and allowed the svg to work under test. Since the source code is now
bundler independent, we need to tell cypress to bundle the svgs under
test. To do this, we pass our webpack config into
@cypress/webpack-preprocessor to allow loading the svgs. This is not a
breaking change.
* chore: switch from hash to contenthash
* chore: remove use of moduleIds for chunkIds as the defaults are likely better and handle caching/debugging better
* chore: mark tests that need to be updated as node 12 and 14 are no longer supported. node 12 doesnt ship with fs/promises, which causes fatal errors with babel-loader 9
try to test against new docker images [run ci]
* chore: updating v8 snapshot cache
* chore: updating v8 snapshot cache
* chore: updating v8 snapshot cache
* skip http/https form submittal unit tests for experimental webkit due to the clientRoute 404ing after submitting
* chore: pin webpack 5 for component tests in cy in cy to prevent using 4 from hoisted deps [run ci]
* build binaries [run ci]
* chore: bump circle cache [run ci]
* fix: turn off svgo from @svgr/webpack to preserve our scss class names within the svg. see https://react-svgr.com/docs/options/#svgo [run ci]
---------
Co-authored-by: cypress-bot[bot] <+cypress-bot[bot]@users.noreply.github.com>
* chore: update darwin v8 snapshot
* BREAKING CHANGE: set video to false by default (system tests need updating).
* Update cli/CHANGELOG.md
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* chore: update type comments
* chore: update protocol snapshot
* run ci
* run ci
* set video to true for chrome browser crash test
* chore: put in workaround for failing system test spec to be fixed in 27062
* chore: allow retries on actionability tests to be at least one retry as the CI tests run faster without video on
* chore: fix flaky navigation test where done is called multiple times almsot always, but sometimes throws an error
---------
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* feat: Selective CSP header directive stripping from HTTPResponse
- uses `stripCspDirectives` config option
* feat: Selective CSP header directive permission from HTTPResponse
- uses `experimentalCspAllowList` config option
* Address Review Comments:
- Add i18n for `experimentalCspAllowList`
- Remove PR link in changelog
- Fix docs link in changelog
- Remove extra typedef additions
- Update validation error message and snapshot
- Fix middleware negated conditional
* chore: refactor driver test into system tests to get better test
coverage on experimentalCspAllowList options
* Address Review Comments:
- Remove legacyOption for `experimentalCspAllowList`
- Update App desc for `experimentalCspAllowList` to include "Content-Security-Policy-Report-Only"
- Modify CHANGELOG wording
- Specify “never” overrideLevel
- Remove unused validator (+2 squashed commits)
- Add "Addresses" note in CHANGELOG to satisfy automation
- Set `canUpdateDuringTestTime` to `false` to prevent confusion
* chore: Add `frame-src` and `child-src` to conditional CSP directives
* chore: Rename `isSubsetOf` to `isArrayIncludingAny`
* chore: fix CLI linting types
* chore: fix server unit tests
* chore: fix system tests within firefox and webkit
* chore: add form-action test
* chore: update system test snapshots
* chore: skip tests in webkit due to form-action flakiness
* chore: Move 'sandbox' and 'navigate-to' into `unsupportedCSPDirectives`
- Add additional system tests
- Update snapshots and unit test
* chore: update system test snapshots
* chore: fix system tests
* chore: do not run csp tests within firefox or webkit due to flake issues in CI
* chore: attempt to increase intercept delay to avoid race condition
* chore: update new snapshots with video defaults work
* chore: update changelog
---------
Co-authored-by: Bill Glesias <bglesias@gmail.com>
Co-authored-by: Matt Schile <mschile@cypress.io>
* chore: allow users to pass true to videoCompression config and only allow valudes 1-51 inclusively to be passed in
* run ci
* chore: allow zero to be option for CRF as we will coerve it to false and skip compression to have a lossless video, which has the same effect
* Update cli/CHANGELOG.md
* chore: update videoCompression types
* chore: update changelog
* Update cli/CHANGELOG.md
* Update cli/types/cypress.d.ts
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* Update packages/config/src/validation.ts
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* chore: update config snapshots
* Update packages/config/test/validation.spec.ts
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* chore: add system test on videoCompression=true coersion
* chore: document 0 as being false and not a valid CRF option for cypress video compression and make CRF valid values 1-51
* chore: fix types
* chore: fix types
* chore: fix change to log as feature and not bugfix
* chore: fix server side unit tests
---------
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
* chore: add Nx Cloud (#26712)
* chore: add empty nx.json [run ci]
* chore: add nx cloud runner [run ci]
* chore: add nx-cloud dep [run ci]
* chore: update local nx cloud accessToken to be read-only
* feat: update git related messages for runs and debug (#26758)
* feat(app): update DebugError copy
* feat: set isUsingGit project flag and consume in DebugContainer
* feat(app): update not using git icon for DebugError
* feat(app): displays alert on runs page when not using git
* feat(app): add component for when no runs for current branch
* feat(app): add warning for no runs for branch on runs container
* chore: add feat to CHANGELOG
* chore: remove logged status value
* chore: resolve import from merge conflict
* test(app): stub branch name for e2e runs spec
* chore: add line break in changelog entry
* chore: cleanup PR
* chore: fix i18n import for DebugBranchError
* chore: add utm and update Warning links to inline
* chore: capitalize Git in i18n
* ref: warning liink
* test: add i18n to tests
* test(app): change utm_source assertions
* chore: cleanup pr
* chore: remove unused prop
* test(app): remove no git warning when moving to runs page in e2e
* chore: change template logic
* chore: remove duplicate RUNS_TAB_MEDIUM const
* Changed Debug test assertion and reordered new components for Debug
---------
Co-authored-by: Stokes Player <stokes.player@gmail.com>
* chore: rename video processing events to capture/compress (#26800)
* chore: change processing nomenclature to compressing when printing the run.
* chore: rename 'capturing of' to 'capturing'
* chore: rename upload results to upload screenshots & videos (#26811)
* chore: rename upload results to upload screenshots & videos
* run ci
* chore: capture versions of relevant dependencies with `x-dependencies` header (#26814)
* chore: update changlelog script to handle revert pr ref (#26801)
* fix: Correct typescript scaffold dependency (#26815)
* fix: correct typescript scaffold dependency (#26204)
* add changelog
* Update change log for PR comment
Co-authored-by: Mike Plummer <mike-plummer@users.noreply.github.com>
---------
Co-authored-by: Mike Plummer <mike-plummer@users.noreply.github.com>
Co-authored-by: Mark Noonan <mark@cypress.io>
* chore: 12.13.0 prep (#26833)
* chore: 12.13.0 release (#26834)
* chore: 12.13.0 changelog fix
* remove pending, bump version
* fix typo
* chore: release @cypress/vite-plugin-cypress-esm-v1.0.1
[skip ci]
* chore: Implement runSpec mutation (#26782)
* chore: replace gitter badge with discord on readme (#26771)
* chore: add GraphQL mutation for sending system notifications via Electron (#26773)
* fix: upgrade typescript from 4.7.4 to 4.9.5 (#26826)
Snyk has created this PR to upgrade typescript from 4.7.4 to 4.9.5.
See this package in npm:
See this project in Snyk:
https://app.snyk.io/org/cypress-opensource/project/d5b36925-e6ee-455d-9649-6560a9aca413?utm_source=github&utm_medium=referral&page=upgrade-pr
* test: fix 2 broken tests for Windows (#26854)
* Update stale_issues_and_pr_cleanup.yml
Upped the number of operations per run. Have been manually doing that so this job can get through all the issues in the repo with no problems.
* chore(dep): [Snyk] Upgrade vite from 2.9.13 to 2.9.15 (#26830)
Co-authored-by: Ben M <benm@cypress.io>
* Update triage_add_to_project.yml
* chore: fix minor background color styling in debug results component (#26887)
* Update stale_issues_and_pr_cleanup.yml
stalebot was incorrectly configured to run in debug mode. I have updated the default to run in normal mode when running scheduled
* chore: Deprecate @cypress/xpath package (#26893)
* chore: add telemetry realworld app (#26896)
* chore: capture telemetry for realworld app maybe
* idk what i was doing
* setup record key and telemetry
* testing
* override project id
* some times we just need a little context.
* Adding tests
* Adding comment
* fix yarn lock
* Trying this....
* fix tests?
---------
Co-authored-by: Jordan <jordan@jpdesigning.com>
Co-authored-by: Stokes Player <stokes.player@gmail.com>
Co-authored-by: Bill Glesias <bglesias@gmail.com>
Co-authored-by: Adam Stone-Lord <adams@cypress.io>
Co-authored-by: Emily Rohrbough <emilyrohrbough@users.noreply.github.com>
Co-authored-by: Dave Kasper <dave.m.kasper@gmail.com>
Co-authored-by: Mike Plummer <mike-plummer@users.noreply.github.com>
Co-authored-by: Mark Noonan <mark@cypress.io>
Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com>
Co-authored-by: semantic-release-bot <semantic-release-bot@martynus.net>
Co-authored-by: Ely Lucas <ely@meta-tek.net>
Co-authored-by: Snyk bot <snyk-bot@snyk.io>
Co-authored-by: Stokes Player <stokes@cypress.io>
Co-authored-by: Ben M <benm@cypress.io>
Co-authored-by: Jennifer Shehane <jennifer@cypress.io>
* feat: set up experimentalUseDefaultDocumentDomain to disallow document.domain overwritting
* use default domain around experimentalUseDefaultDocumentDomain in main iframe and spec bridge iframes. Also adapt CORS policy to use same-origin if experimental flag is set
* run ci
* fix: add insertion of experimental flag where is was needed/missing
* chore: add system test to exercise experimental flag for expected behavior
* fix: fix issues with template updates to conform to squirrelly v7
* fix: update config tests to include new experimental flag
* run ci
* fix: trailing whitespace [run ci]
* chore: update snapshot
* run ci
* fix: update proxy unit tests to account for experimentalUseDefaultDocumentDomain
* run ci
* fix: Allow component tests with special characters in filepath (#25299)
feat: cut over experimental flag to take list of known problematic domains via string/glob pattern
run ci
chore: update system test and fix broken config
* fix: fix server unit and integration tests. integration tests should no longer use google to test against injection as we do not inject document.domain on google domains
* run ci
* run ci
* fix: server integration tests where google documents are expected to receive document.domain injection. Kept test same by changing URL
* run ci
* fix: update server test with mssing unupdated assertions
* run ci
* fix: turn off experimental flag by default while recommending sane defaults to users to configure
* run ci
* chore: fix typings [run ci]
* run ci
* chore: make experiment an e2e option only
* run ci
* chore: address comments in code review
* chore: rename experimentalUseDefaultDocumentDomain to experimentalSkipDomainInjection
* fix regression in shouldInjectionDocumentDomain utility function and add unit tests
* run ci
* chore: rename documentSuperDomainIfExists to superDomain [run ci]
* chore: address comments from code review
* chore: just pass opts through to policyForDomain
* run ci
Co-authored-by: Mike Plummer <mike-plummer@users.noreply.github.com>
* feat: re-enable Run All Specs (#24683)
* fix: disable query in run-mode
* feat: Run-all-specs-ui (#24738)
* wip run-all-specs-ui
* added component tests for specs-list, inline-specs-list, and run-all-specs
* updated tests for specslist, inlinespeclist, and runAllSpecs
* simplify test
* make prop with default value optional
* enable run all specs
* use named slot for clarify
* use Record type
* remove un-necessary dynamic component
* use group and hover to inline css
* fix tests
* fix tests and use snapshot for style testing
Co-authored-by: Lachlan Miller <lachlan.miller.1990@outlook.com>
* fix typescript
* fix test
* build binary for run all specs
* fix: tweaks for Run All Specs (#24746)
* update ui [skip ci]
* change types to reflect run all specs is e2e only
* chore: separate run all specs components
* fix typescript error
* fix: run-all UI tweaks and keyboard nav (#24757)
Closes undefined
* fix UI test
* move listener and use i18n [skip ci] (#24761)
* fix: run-all-specs duplicate file-matching (#24763)
* chore: limit linting [skip ci]
* do not get separator until app is loaded
* fix test
Co-authored-by: Zachary Williams <ZachJW34@gmail.com>
Co-authored-by: amehta265 <65267668+amehta265@users.noreply.github.com>
* fix: remove server and route commands
* This won't totally pass but it's a step in the right direction.
* Should be down to only two failing tests due to abort.
* fix the alias.<thing> api for intercept to be equivalent to route.
* adjust timeouts to reduce flake, ideally
* skip xhr abort tests
* More clean up
* Increase timeout
* trick next version
* fix unit tests probably
* yeah sure
* Apply suggestions from code review
Co-authored-by: Blue F <blue@cypress.io>
* remove addXhrLog
* Remove more usage of XHR stemming from proxy-logging
* Remove a couple of files that found their way back with the merge from release/12.0.0
* remove more server types
* increase timeouts again to prevent flake
Co-authored-by: Blue F <blue@cypress.io>
Co-authored-by: Zach Bloomquist <git@chary.us>
Lachlan Miller