Commit Graph

62 Commits

Author SHA1 Message Date
Kukhyeon Heo
e22b362d4c Out-of-the-box TypeScript Support (#5906)
* Out-of-the-box TypeScript support for plugins

* Out-of-the-box TypeScript Support for tests.

* Fix server unit tests.

* Don't use bundled typescript.

* Added esModuleInterop option.

* Fixed unit tests.

* Added simple_tsify to integration test.

* Added integration tests for ts spec files.

* simple_tsify to ignore transpiling json files.

* transpile on flush.

* Fix lint failure.

* Added error message.

* Add error message for restart test runner for support file.

* Added JSDoc comment.

* Remove preset-typescript.

* Fix integration test failure.

* Fix type errors.

* Add e2e test for typescript

* Fix lint error.

* Add plugin support and e2e test.

* Fix server unit test failures.

* Update snapshots.

* Migrate typescript compilation code to browserify-preprocessor

* Remove unnecessary files and test.

They're tested in preprocessor.

* Fix test failures.

* Update @types/react.

* Update yarn.lock.

* Clear name: registerTS -> tsRegistered.

* Clarify why tsRegistered = true exists.

* Add working examples for support and plugins.

* Move TypeScript tests to its own file.
2020-04-13 21:39:13 +06:30
Jennifer Shehane
2ba53f6837 Better handle reserved key CYPRESS_ENV being set by users to va… (#6437)
* Add warning when setting CYPRESS_ENV to non-production value

* Add warning and update error when setting CYPRESS_ENV in config to non-production value

* Update config test/to throw

* we want warning, not throw

* Rename env var to CYPRESS_INTERNAL_ENV + fix warning to actually warn when staging

* update cli snapshot to include new 'info' command

* yarn.lock

* removed the warning from config, is overboard on our own tests 😓

* cleanup from review

Co-authored-by: Gleb Bahmutov <gleb.bahmutov@gmail.com>
2020-03-06 16:53:48 +06:30
Chris Breiding
e93e6ae0b4 Validate plugin event registration (#5356)
* Turn exception message into something human readable

* Pass ipc as parameter to invoke function

* Creating file to validate event name and handler

* Creating tests to validate_event

* Remove ipc from invoke parameter

* Removing ipc parameter being passed to validateEvent

* convert spec to js

* increase line-height for plugins error message

* refactor error messages and implementation

* fix race condition where async error in plugins file could hang run

a quick async error at the root of the plugins file had the potential to hang the run because the ‘exitEarlyWithErr’ listener was registered later than that error was emitted

this enables that error to be tracked so we can properly exit at the appropriate time

it also refactors run.js to not rely on an event emitted on the project and instead passes through an onError handler, which makes more sense since the event was only used in run.js (except for one case). it also makes for easier unit testing

* fix missing reference

* fix duplicate reference

* fix args being passed in incorrectly

* fix way args were handled in server.open

* fix exit early implementation

* fix duplicate logging

* fix unit test

* update snapshot

* fix missing reference

* add e2e test to cover plugin registration validation

* clean up after merge

* add back snapshot

* fix e2e tests

Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com>
Co-authored-by: Zach Bloomquist <github@chary.us>
Co-authored-by: Jennifer Shehane <shehane.jennifer@gmail.com>
2020-03-05 16:00:42 -05:00
Chris Breiding
474b80a50f Fix race condition when there's an early asynchronous error in… (#6610)
* fix race condition when there's an async error in root of plugins file

* return the promise

* fix routes creation

* fix tests

* fix error throwing and add tests

* update snapshots

* revert changes to server.open signatures in tests

* fix test

* properly wrap error so it doesn't log twice

* slow down test to ensure plugins error occurs before run is over

* wait to log early exit error until after run start
2020-03-04 11:59:01 -05:00
Zach Bloomquist
a4b11e1928 Use CDP to record video for Electron, record video in Electron… (#6632)
* Use CDP to record video for Electron

* Enable video recording in Electron headed

* Test video in headed + headless for all browsers

* Update e2e tests
2020-03-03 15:42:43 -05:00
Jennifer Shehane
b92bd29571 Fix typos in error messages (#6508) 2020-02-19 23:17:41 +06:30
Zach Bloomquist
ca46c1606f Handle errors during initial Marionette connection (#6370)
* use Promises driver and handle connection errors

* repin marionette-client dep

* --compilers... so our tests run....

* update firefox spec

* fix failing specs from tests not running

* update error with a hint

* fix the other on

* tests
2020-02-07 17:27:45 -05:00
Chris Breiding
d76123b3d9 Add Firefox support (#1359)
Co-authored-by: Ben Kucera <14625260+Bkucera@users.noreply.github.com>
Co-authored-by: Gleb Bahmutov <gleb.bahmutov@gmail.com>
Co-authored-by: Brian Mann <brian.mann86@gmail.com>
Co-authored-by: Zach Bloomquist <github@chary.us>
Co-authored-by: Jennifer Shehane <shehane.jennifer@gmail.com>
2020-02-06 10:52:13 -05:00
Zach Bloomquist
291a3546da Remove --browser canary backwards-compatibility (#6333)
* remove --browser canary compatibility

* update warning when canary is passed

* add tests for --browser canary warning

* Update packages/server/test/unit/browsers/browsers_spec.coffee
2020-02-05 15:53:52 -05:00
Ben Kucera
94176149c4 deprecated before:browser:launch event (#6293)
* deprecate old API for before:browser:launch

* trigger ci

* allow chrome preferences to be overwritten in before:browser:launch

* add e2e test for chrome preferences

* add launchOptions.windowSize, refactor plugin handling into browser/utils

* async/await-ify chrome.ts, update e2e.js

* add padding to warning, allow more than 1 warning in run mode

* remove psInclude

* bump snap-shot-it, remove patch-package

* fix failing tests, tighten up code

* remove unused Promise.all

* more code cleanup from coffeescript conversion

* fix expectedExitcode

* update snapshot with cross origin error message

* more code cleanup from coffeescript conversion

* remove global state, cleanup normalizing stdout in snapshots

* fix asserting on the expected exit code

* remove firefox a default browser in e2e tests

* remove dupe const

* make onStdout return first arg by default

* remove only

* make expectedExitCode: 0 the default, remove duplicate option from all e2e tests

* fix test for electron

* reuse _getArgs() properly, tighten up dupe code, move perf e2e tests to use new e2e.it helpers

* make e2e test pass properly, and capture snapshot

- this was being overlooked because expectedExitCode wasn’t being set
(until now its the default)

* revert firefox specific changes to reduce PR diff

* remove newline

* temp 02/04/20 [skip ci]

* add more scenarios / return values

* rename plugin deprecation project fixture

* prevent warnings in run mode from being called multiple times in the same browser launch

- add plugin integration tests
- cleanup some promise code in browsers/chrome
- refactor the e2e deprecation tests

* update e2e deprecation specs, test that deprecation warning is displayed once per spec on a run

* cleanup let -> const, coffescript conversion

* bump snap-shot-it, remove patch-package

(cherry picked from commit ba23be5349)

* bump cli snap-shot-it

* cleanup: let -> const, parseFloat, formatting

* revert cross origin normalization

* fix spacing

* add types for plugin events

* dont pad warning message in run mode

* add e2e test for adding extensions before:browser:launch

* fix failing electron unit test, consolidate creating default launch options

* add extensions for electron + warning if install fails

* fix failing tests, yield the right before:browser:launch args signature

* e2e test electron via devtools extension

* remove .only

* add stdout assertions to e2e/1_deprecated_spec

* remove snapshot whitespace

* rename deprecation event

* update deprecated browser launch args warning message

* throw error on unexpected bbl property

* add tests for warning + error on bbl

* update snapshot

* revive 2 useful tests to validate how launchOption args are merged

* try fix e2e fullscreen spec, update snapshots

* tighten up code for throwing errors on unknown launch options properties

- list out the unknown and expected properties using existing
conventions
- tighten up the error message a bit

* rename options -> launchOptions, add e2e test for adding unknown properties to launchOptions, removed dupe snapshots

* fix fullscreen e2e test

* only push user gesture arg if chromium

* add e2e tests for throwing + rejecting errors, run e2e tests on all browsers, avoid ps logic only in electron

* remove conditional args in fullscreen test

* dont automatically install the latest version of chrome in circle

* add stubs for new electron properties

* switch from using port 5555 to 5544 to avoid common conflicts

* temporarily commenting out windowSize launchOption so release is unblocked and all tests pass

- can add it back in later once the e2e tests pass in CI

* make the path to chrome profile correctly dynamic to account for all operating systems

* remove magic length(8) and slice out the first 4 custom args set from the plugin launch options

* add back --start-maximized in chrome

* deleting windowSize-related code... for now!

Co-authored-by: Zach Bloomquist <github@chary.us>
Co-authored-by: Brian Mann <brian.mann86@gmail.com>
Co-authored-by: Chris Breiding <chrisbreiding@users.noreply.github.com>
2020-02-05 13:07:59 -05:00
Gleb Bahmutov
76a18307f0 throw specific error when cannot parse Cypress arguments (#6280)
* WIP: throw specific error when cannot parse Cypress arguments

* test invalid config string

* add unit test

* rework error message, handle env, reporter options and config

* add args error unit tests

* rework snapshots

* use reporterOptions name

* update snapshot
2020-01-31 22:34:47 -05:00
renovate[bot]
c3ad1a372a fix(deps): update dependency ansi_up to version .x 🌟 (#4331)
* fix(deps): update ansi_up to 4.0.3 🌟

* Update ansi_up to use new non-deprecated 2.0+ api

* Do not pass 'use_classes: true' (defaults to false) - we are expecting rgb values in our tests

* set back to use_classes

* Fix 'use_classes' option which was being incorrectly passed

Co-authored-by: Renovate Bot <renovatebot@gmail.com>
Co-authored-by: Jennifer Shehane <shehane.jennifer@gmail.com>
2019-12-20 18:31:15 +06:30
Zach Bloomquist
35109fb08c Chrome headless (#5676)
* always disable xvfb

* add --headless

* Revert "always disable xvfb" - it is still needed for Electron

This reverts commit 058679f4ce.

* updates

* update 3_plugins_spec

* update wording now that chrome can be headless

* fix video recording when headless

* Don't assume that chrome is headed

* update electron video recording message

* Add 2_headless_spec for Cypress.browser values

* update headless language

* still use headless by default for electron and headed for chrome

* fix up cli

* fix e2e tests

* update npm api types

* fix 2_headless_spec

* keep alphabetical ordering

* increase binary size limit

* add a comment on the cli error impl

* use _.defaults for default for headed

* fix

* _.defaults mutates
2019-12-12 12:26:32 -05:00
Jennifer Shehane
692a3c07d9 Support --tag argument (#5164)
* Alphabetize cli options for my own sanity

* begin added tag flag

* Fix some minor grammar in cli help output + be more specific for specs description

* update snapshot based on alphabetization change

* update snapshot to include --tag in help output

* update logic for pulling out space delimited args to look through --tag and --spec flags dynamically

* Support and pass along tag flag to run and record

- show err if passed without record flag
- sanitize args into comma separated string
- display tag in record errors

* fix some tests/snapshots where 'tag' was missing

* Actually try passing in tag through tests to ensure it prints.

* Merge branch 'develop' into issue-2561-tags

# Conflicts:
#	cli/__snapshots__/cli_spec.js
#	cli/lib/cli.js
#	cli/lib/exec/run.js
#	packages/server/lib/modes/run.js
#	packages/server/lib/util/args.js

* Send 'tags' as an array to backend API

* Update json-schemas to query against 2.2.0 of postRun

- this will require a bump to json-schemas repo

* update test to reflect tags array

* update snapshot to display nightly tag

* rearrange args to alphabetical order in specs

* Add tags to runResponses / remove tag from incorrect instance post

* Fix failing specs / snapshots

* Update error messages + snapshots

* Fix snapshot that no longer displays tag arg

* fix args unit test

* remove extra slash

* add a few more cli tests

* another test just in case

* a quick unit test for displayFlags utility


Co-authored-by: Gleb Bahmutov <gleb.bahmutov@gmail.com>
2019-12-05 23:02:14 +06:30
Gleb Bahmutov
b03b25c258 feat: pass list of browsers to plugins file (#5068)
and allow project to customize the list of browsers
2019-11-19 09:02:17 -05:00
Zach Bloomquist
7ef3078f6a Increase CDP timeout to 20 seconds; add logs after 5 seconds (#5610)
* Increase CDP timeout to 20 seconds; add logs after 5 seconds

* update error messages
2019-11-05 14:25:29 -05:00
Zach Bloomquist
a3265ccdef Control Chrome cookies through CDP (#5297)
* try connecting to chrome remote interface

* linting

* print CRI targets for better debugging

* linting

* load empty tab first when connecting to CRI

* first load blank page, then navigate

* Page.navigate is working

* linting

* remove title

* add mocha banner

* more banners

* update some server unit tests

* update integration test

* document how to run single driver spec file

* set the focus back on the page before navigating from blank chrome tab

* update server unit test

* do not store Chrome remote interface reference for now

* record video of the Chrome tab using screencast API

* use dynamic port to connect to Chrome remote interface

* update unit tests

* refactoring

* wrap chrome remote interface in our interface, limit access to send

* resolved merge

* fix reference

* passing run unit spec

* stub canary search for CI to pass

* add build step to packages/server

* update chrome spec

* do not build js on install for server

* updated spec snapshots

* update 6_visit_spec snapshot

* update snapshot for 6_web_security_spec

* update snapshot for 3_plugins_spec

* update snapshot for 3_user_agent_spec

* update snapshot for 5_stdout_spec

* update snapshot for 2_browser_path_spec

* do not git ignore js files, add note why

* update several snapshots with video on Chrome

* update visit performance snapshot

* add chrome-remote-interface dependency

* cleanup coffeescript conversion to JS, fix some type errors, make parallel override clearer

* fix failing tests

* Fix snapshot - now we do record in Chrome, so warning message is no longer there.

* remove chrome warnings about not recording from snapshot

* Remove performance tests from 6_visit_spec snapshot

* Remove error from snapshot

* Add newline back to cy_visit_performance_spec snapshot

* Use CDP to control Chrome cookies + screenshot

* Add devtools types

* Cleanup

* Cleanup

* Add guards for minimum CDP version

* Fix failing tests

* Split cdp_automation_spec out of electron_spec

* Move takeScreenshot to cri-client

* Navigate to about:blank

* look for blank page url

* add note about avoiding Prettier

* disable prettier a little more

* call chrome remote interface close after each spec

* return promise when starting screencast

* update failing unit tests, add cri client close test

* update integration test

* Add verbose debug statements to cri-client

* Use connect.createRetryingSocket for CDP connection

* record video from chrome browsers

* add method for validating browser family

* update e2e spec snapshot

* update 4_request_spec snapshot

* update snapshot for spec 1_commands_outside_of_test_spec

* update snapshot for 3_plugins_spec

* update snapshot for spec 3_user_agent_spec

* try: Always log video capturing errors

* update snapshot for 2_browser_path_spec

* update snapshot for 2_cookies_spec

* better browser family test

* update snapshot for 5_stdout_spec

* update snapshot for 5_subdomain_spec

* Add protocol_spec tests

* do not capture video during performance test

* Add test for VIDEO_POST_PROCESSING_FAILED warning

* Add basic cookie validation in cy.setCookie

* Update cdp_automation to throw on Network.setCookie failure code

* Update tests 🎉

* Update snapshot

* Fix test

* Remove redundant logs, cleanup

* Add cri-client_spec, fix some small bugs, improve errors

* Update dep

* use client.on to register screencast callback

* use isCookieName

* strict-cookie-parser@3.1.0

* cleanup prettier, extract some functions, switch to browser.family

* moar cleanup and fixes

* add logging to the cri-client so we can see every message sent + received to the cdp protocol

* bump bluebird to 3.7.0 for .tapCatch addition

* Fix unit tests

* WIP: update e2e test to ensure that duration of the video matches what we expect

* Test duration of recorded video

* Run 6_video_compression in chrome + electron

* Cleanup

* finish ffmpeg duration verification

* Update 8_reporters_spec snapshot

* Fix cri-client test

* Update CRI close logic to monkey-patch browser.kill

* add isBrowserFamily back

* make it possible for remote-debugging-port to get overridden

* Make CDP timeout 5s; add unit, e2e tests for CDP failure; add user-friendly CDP failure error

* Update tests

* Use CYPRESS_REMOTE_DEBUGGING_PORT to set CDP port; update CDP error message

* Change new Buffer to Buffer.from

* Apply name validation on all cookie commands

* Just throw on Chrome start if the CDP version is < 1.3

* Fix cypress_spec

* Use CDP to set resolution + scale factor in Chrome e2e

* Revert "Use CDP to set resolution + scale factor in Chrome e2e"

This reverts commit a1b86d9413.

* use CYPRESS_FORCE_BROWSER_SCALE to force standard resolution

* don't do --window-size --kiosk

* Use CDP to set resolution + scale factor in Chrome e2e

* Revert "use CYPRESS_FORCE_BROWSER_SCALE to force standard resolution"

This reverts commit 22c5e78670.

* Use Page.captureScreenshot for Electron + Chrome, reduce logic

* Use before() task to force device metrics in Chrome

* Fix protocol_spec

* Update 7_record_spec to allow for before() hook

* Update 6_task_spec snapshot

* Appease eslint

* Update hooks in 5_spec_isolation snapshot

* some general promisification and cleanup

* feedback on pluginsfile

* cdp_automation feedback

* chrome.coff feedback

* feedback

* run e2e tests on port 4466, ensure no e2e test ever runs on 5566 to prevent conflicting with debugger port

* accept new 'remote:debugger:protocol' automation command to control device metrics overrides

* update web security e2e to run on electron + chrome

* run web security tests in electorn, disable context isolation

* pass disable-site-isolation-trials to Electron so webSecurity works

* Fix errors in e2e tests caused by extra log item

* fix cri-client unit tests

* fancy arrows in log message


Co-authored-by: Gleb Bahmutov <gleb.bahmutov@gmail.com>
Co-authored-by: Brian Mann <brian.mann86@gmail.com>
Co-authored-by: Jennifer Shehane <jennifer@cypress.io>
2019-10-21 17:07:53 -04:00
Gleb Bahmutov
a877474df1 Connect to Chrome Remote Interface when launching Chrome and re… (#4628)
* try connecting to chrome remote interface

* linting

* print CRI targets for better debugging

* linting

* load empty tab first when connecting to CRI

* first load blank page, then navigate

* Page.navigate is working

* linting

* remove title

* add mocha banner

* more banners

* update some server unit tests

* update integration test

* document how to run single driver spec file

* set the focus back on the page before navigating from blank chrome tab

* update server unit test

* do not store Chrome remote interface reference for now

* record video of the Chrome tab using screencast API

* use dynamic port to connect to Chrome remote interface

* update unit tests

* refactoring

* wrap chrome remote interface in our interface, limit access to send

* resolved merge

* fix reference

* passing run unit spec

* stub canary search for CI to pass

* add build step to packages/server

* update chrome spec

* do not build js on install for server

* updated spec snapshots

* update 6_visit_spec snapshot

* update snapshot for 6_web_security_spec

* update snapshot for 3_plugins_spec

* update snapshot for 3_user_agent_spec

* update snapshot for 5_stdout_spec

* update snapshot for 2_browser_path_spec

* do not git ignore js files, add note why

* update several snapshots with video on Chrome

* update visit performance snapshot

* add chrome-remote-interface dependency

* cleanup coffeescript conversion to JS, fix some type errors, make parallel override clearer

* fix failing tests

* Fix snapshot - now we do record in Chrome, so warning message is no longer there.

* remove chrome warnings about not recording from snapshot

* Remove performance tests from 6_visit_spec snapshot

* Remove error from snapshot

* Add newline back to cy_visit_performance_spec snapshot

* Navigate to about:blank

* look for blank page url

* add note about avoiding Prettier

* disable prettier a little more

* call chrome remote interface close after each spec

* return promise when starting screencast

* update failing unit tests, add cri client close test

* update integration test

* Add verbose debug statements to cri-client

* Use connect.createRetryingSocket for CDP connection

* record video from chrome browsers

* add method for validating browser family

* update e2e spec snapshot

* update 4_request_spec snapshot

* update snapshot for spec 1_commands_outside_of_test_spec

* update snapshot for 3_plugins_spec

* update snapshot for spec 3_user_agent_spec

* try: Always log video capturing errors

* update snapshot for 2_browser_path_spec

* update snapshot for 2_cookies_spec

* better browser family test

* update snapshot for 5_stdout_spec

* update snapshot for 5_subdomain_spec

* Add protocol_spec tests

* do not capture video during performance test

* Add test for VIDEO_POST_PROCESSING_FAILED warning

* use client.on to register screencast callback

* cleanup prettier, extract some functions, switch to browser.family

* moar cleanup and fixes

* add logging to the cri-client so we can see every message sent + received to the cdp protocol

* bump bluebird to 3.7.0 for .tapCatch addition

* Fix unit tests

* WIP: update e2e test to ensure that duration of the video matches what we expect

* Test duration of recorded video

* Run 6_video_compression in chrome + electron

* Cleanup

* finish ffmpeg duration verification

* Update 8_reporters_spec snapshot

* Update CRI close logic to monkey-patch browser.kill

* add isBrowserFamily back

* make it possible for remote-debugging-port to get overridden

* Make CDP timeout 5s; add unit, e2e tests for CDP failure; add user-friendly CDP failure error

* Update tests

* Use CYPRESS_REMOTE_DEBUGGING_PORT to set CDP port; update CDP error message


Co-authored-by: Brian Mann <brian.mann86@gmail.com>
Co-authored-by: Jennifer Shehane <jennifer@cypress.io>
Co-authored-by: Zach Bloomquist <github@chary.us>
2019-10-11 18:36:05 -04:00
Gleb Bahmutov
4ec77e3d58 use Node path from config file to run plugins (#4436)
* use Node path from config file to run plugins

* add finding system node

* memoize node find, use fix-path if Node is not found

* find Node asynchronously

* update note on ENFILE

* print node version before executing registered event

* use nodeVersion option, add validator isOneOf

* linting

* remove resolve-dir

* debugging unit test

* resolve original promise to make the tests run correctly

* update second test

* add unit tests

* upgrade snap-shot-it package to be able to update snapshots

* make finding node path part of startup, merge into config

* update tests

* add node version & path to run output

* add node version display to desktop-gui

* add warnings, add tests, add learn more link in desktop-gui

* Revert "add node version & path to run output"

This reverts commit 40a3218175.

* only show Node Version if it is non-default

* Add e2e test for using system node

* cleanup

* add tests that resolvedNodePath and Version are available in tests

* assert nodeVersion == system

* add nodeVersion to schema

* add new configoptions to types

* add fallback if pluginsfile is disabled

* new package.json sorting

* update some of the wording/styling of Node Version panel

* remove ` from schema file

* clean up decaffeinated tests + update wording when no pluginsfile is present

* playing with messaging.

* push updated node version panel design

* update design + copy of Node panel again

* some more iteration on design (WIP)

* Finish design changes + fix tests

* add debug message

* linting problems

* add missed require

* Add comment to refactor the colWidths in this PR after another PR goes

* replace the exact length of Node version in specfiles

* print object for node version and path instead of %s %s

* update snapshot to match v12 length


Co-authored-by: Zach Bloomquist <github@chary.us>
Co-authored-by: Jennifer Shehane <jennifer@cypress.io>
2019-10-01 17:08:11 -04:00
Zach Bloomquist
49f5b3e80c Introduce --config-file argument (#3246)
* cli, server: introduce --config-file argument

* server: remove unused import

* server: wip

* server: consider --config-file in settings

* server: pass options to settings.read from config

* server: store options in Project class, pass to all settings calls

* server: _initPlugins needs to accept options, for being called from server

* server: accept optional external options in open

* cli: update help snapshots

* server: realizing now that these were written like this so they could be stubbed - removing some unnecessary usages of @options

* cli: pass configFile when it's false

* server: --config-file false and --config-file blah.json work

* server: add unit tests for --config-file

* server: pass configFile to desktop-gui

* desktop-gui: display 'cypress.json' according to --config-file arg

* desktop-gui: add integration tests for --config-file

* cli: add tests for --config-file

* PR changes

* PR changes

* cli: update snapshots

* server: updating error messages

* runner: update cypress.json mention

* fixing name overlap

* server: integration tests for --config-file

* runner: update Header component tests

* cli: fix snapshot

* desktop-gui: fix test

* driver: fixing error messages - not really any visibility to cli args from here so just static strings

* server: update snapshots

* server: update snapshots

* cli: updating snapshot

* driver: how did i miss this?

* add skipped blank line to the snapshot

* fix missing proxy require statement (was lost in merge of develop)...weird

* add module API defs to types

* module API tests

* send better error when config file can't be found

* fix dtslint test

* update cli help to use 'configuration file'

* update snapshot using 7.7.1 in place

* fix failing config_spec

* be.visible

* show custom config file name in driver errors

* add tests for non-default config file in driver error messages

* single-quote config file name

* 🙅 IIFEs 🙅

* 🤦

* fix failing test

* fix failing test, cleanup

* lint

* delete duplicate coffee spec

* Update run.js

* Delete app_spec.js.mp4

* in open mode, only store projects to recents list if 'cypress.json' is the configFile

discussion: https://git.io/JeGyF

* feedback
2019-09-27 10:25:07 -04:00
Gleb Bahmutov
9f082d97ca Catch env variable with reserved name CYPRESS_ENV 1621 (#1626)
* server: check CYPRESS_ENV variable when merging configs

* catch invalid CYPRESS_ENV value in CLI, close #1621

* linting

* sanitize platform in test snapshot

* linting

* update error message text

* add missing comma

* fix finally merge in JS code

* pass CLI linter

* fix log reference, should be debug

* use correct sinon reference

* update message, show first part in red

* update error message text
2019-09-23 11:59:49 -04:00
Jennifer Shehane
a7dfda9865 Show warning when binary is run directly (outside npm module) (#4701)
* commit previous changes made by @bahmutov for #1573

* Pass '--run-from-cli' flag through ping test in order to prevent warning from printing

* woops, require 'argsUntil'

* 'headed' was changed to 'interactiveMode'

* fix duplicate misspelled require

* tighten up args utils and cleanup decaf garbage

* cleanup errors.log to take a cb and conditionally be async

* remove extraneous --run-from-cli argument, consolidate to use --cli

- update tests

* fixes tests, ensure that record.createRun() always returns a promise

* refactor tests to handle new errors.logException interface

* make logException always return a promise, cleanup interface, add test

* fix linting errors


Co-authored-by: Brian Mann <brian.mann86@gmail.com>
2019-07-29 07:33:33 -04:00
Jennifer Shehane
f189d5e408 remove packages that do not appear to be used (#4755)
- I may have broken things - let’s see.
2019-07-25 11:08:01 +06:30
Zach Bloomquist
f89a8236a8 Do auth in the user's browser, do auth through the dashboard, s… (#3616)
* desktop-gui: use authBegin api

* server: add auth lib for in-browser auth

* server: unneeded

* server: send proper cors header, actually receive authinfo

* desktop-gui: DASHBOARD_LOGIN

* server: send both auth flows through dashboard login

* server, desktop-gui: exchange code for token, move logic out of desktop gui, cache auth urls

* server: cleanup

* server: refresh token [wip]

* server: cleanup

* server: focus main window on login

* server: focus cypress after login

* server: fix

* server: use id_token so Google Login works, fixes to fallback electron auth flow

* desktop-gui: "Log In with GitHub" -> "Log In to Dashboard"

* server: work without electron

* server: what a bizarre function signature, but ok

* server: unit test for new auth.js

* driver, server: cleanup

* server: use redirects instead of XHR

* server: update api spec

* add some tests for token refreshin

* desktop-gui: update tests, cleanup

* unit tests

* add user unit tests

* server: rely on dashboard to set us up with access_token, user profile, etc

* server: cleanup

* redirect errors too

* server: don't worry about refresh tokens in TR

* guard against null server

* don't regenerate state

* fix auth unit test

* fall back to electron auth if native auth never opens

* break out MarkdownRenderer component

* warn if browser not opened, offer copyable url

* remove electron fallback and all login window handling

* update tests

* send more feedback when authing

* add success message when logging in

* update tests to expect continue button, warnings

* use :contains

* send machineId, version, platform, arch with login

* createSignout -> createLogout, get logoutUrl from /auth v2 endpoint

* Change version queryParam to cypressVersion, keep platform as platform, don't bother sending arch

* Change " Opening browser..." button to display " Browser failed to open"

* "You are now logged in to the Cypress Dashboard as Zach Bloomquist." -> "You are now logged in as Zach Bloomquist."

* POST /signout -> GET /logout

* make fallback URL click-to-select

* add tests for edge cases in browser launching

* cleanup

* logoutUrl -> dashboardLogoutUrl

* getLogout -> postLogout

* getLogout -> postLogout

* send machineId with postLogout
2019-07-09 09:39:10 -04:00
Zach Bloomquist
178e9bf0ab Warn when Cypress detects policies that may interfere with auto… (#4405)
* add chrome policy check that is logged to stdout/displayed in GUI

* improve warning message

* add desktop-gui warning when launching browser

* update tests

* clean up adding warnings to browsers in the gui

* support multiple warnings in gui

* update on-link

* don't show warning in open mode

* add test for warning getting attached, add hyperlink to warning tooltip

* render tooltip warnings with markdown

* pass NO_CHECK_DEPS=1 to skip dep check

* @cypress/react-tooltip@0.5.0 - adds clickability to tooltips

* add link to issue

* dont need to update reporter and runner

* use synchronous console.log, not async errors.log for displaying yellow warnings

* use trigger, do not build up events manually

/cc @flotwig


Co-authored-by: Jennifer Shehane <jennifer@cypress.io>
Co-authored-by: Brian Mann <brian.mann86@gmail.com>
2019-07-03 15:35:58 +02:00
Zach Bloomquist
f74d103a60 Warn on createRun warning from dashboard if warning code is unk… (#4567)
* Warn on createRun if warning code is unknown

* add response validation in 7_record_spec

* add unknown warning fallback test

* update existing tests to match schemas
2019-06-27 15:40:22 -04:00
Jennifer Shehane
cb51d5ed38 update Test Runner to handle API errs for non-private test limits (#4025)
* update Test Runner to handle API errs for non-private test limits

* update record errors to factor in private vs regular tests

- manually reconcile and update all snapshots —______________—


Co-authored-by: Brian Mann <brian.mann86@gmail.com>
2019-05-17 10:36:03 -04:00
Zach Bloomquist
b8a6baaca3 Retry certain requests on failure (#4015)
* retry requests, basic e2e test

* basic e2e test for chrome behavior

* don't use timeouts to test

* some minor cleanup

* validate google chrome's retry behavior w/ a proxy

* get retries on network errors workin

* cleanup

* final changes before switching approach

* Reverting previous approach in request.coffee, server.coffee

* add retryOnNetworkFailure

* now works with retryOnStatusCodeFailure

* retry 4 times in addition to the 1 initial attempt

* add tests for subresources

* much improved error handling

* have the e2e test really work

* e2e baseurl check

* retry baseurl check

* remove special handling for node 8.2.1 bug

* WIP: continue making progress building out request retry

- swap out passthrough’s for pumpify / duplexify / pump
- clean up error handling / messaging / retry logic

* pipe clientrequest events

* buffer req body to disk, restore error handling/retrying

* don't bubble up errors from failed attempts

* actually pipe reqstream, oops

* add some e2e tests for request body

* revert lib/request.coffee to 7be1051

* add almost-working lazy-stream

* manually fire the 'pipe' event on the reqStream to copy headers to the outgoing message

- restore the ‘error’ propagation so that all tests pass for now

* cleanup leaking 'undefined' into stdout, causing failing e2e tests

- do not set onWarning as a property of project, just pass as an
argument

* add new options to request_spec, deduplicate default opts

* use stream buffer in request.coffee

* revert request.coffee

* last stream_buffer changes before removing fs stuff

* remove fs stuff from stream_buffer, add stream piping tests

* it works! :tada::tada:🎉 using duplexify to separate delayStream and reqBodyBuffer

* retry for .5s max on ECONNREFUSED

* add error codes

* don't timeout proxied requests

* restore baseurl check

* update new e2e tests

* make delay work with rp

* propagate clientresponse events

* removing tests that don't do anything now that we don't ESOCKETTIMEOUT on proxied requests

* add new visit, req options to index.d.ts

* don't fail on server-performance-test

* make retries with status code work again

* account for different stack trace in ci

* fix test

* retry https requests

* add tests for https passthru retries working

* clean up error handling for https-proxy

* fix failing https-proxy tests, tweak agent error handling to prevent multiple callbacks

* make expectedExitCode actual vs. expected in the correct order

* bump up e2e test timeout so it can retry and still work

* update tests

* retry up to 500ms on proxied requests

* add tests for incompatible options errors

* remove .only

* maybe this will help it act more consistently

* help e2e test work in ci

* don't reconnect on already made connections

* clarify naming

* wip: testing https proxy

* better debug calls

* WIP: getting proxy passthrough retry tests going

* handle retrying upstream proxy errors

- add tests for successfully retrying proxy errors and for unsuccessful
retries
- fix onClose errors when proxy connection is closed before enough data
is received
- fix not returning setTimeout correctly

* group related code accordingly

* do not build typescript by default, drop extension from main

* more TODO notes

* don't set a default NO_PROXY if NO_PROXY = ''

* debugging-proxy@2.0.0

* null out reqBodyBuffer when req finishes

* wip: retry in agent, not https-proxy [skip-ci]

* update https-proxy to use @packages/network retries

* retry after connection but before proxy tunnel established

* appease my linty overlords

* update https-proxy tests

* update agent specs, decided to still use tls.connect

it's easier to test and has less complexity

* test retrying HTTPS passthru

* debugging-proxy@2.0.1

* increase defaultCommandTimeout 100 -> 200 to prevent flake in CI

* auto formatting

* fix test to be dynamic and not rely on magic constants

* copy types field when linking proxy images, update packages/network types field

* linting

* add network index.js file

* linting

* improve error messaging experience when verifying base url

* only insert 1 new line

* fix failing test not binding to localhost

* removed test that's covered by e2e specs

* remove dash in 're-try'

* some cleanup for readability

* use allocUnsafe per perf

* unset NO_PROXY with an empty string

* move retry ensuring the baseUrl into url, cleanup some imperative code

* if the head is already present during connect, make connection, else wait for first head bytes

* minor formatting, clarity around conditions, naming

* rename retryInterval -> retryIntervals

* set defaults for requests during creation for clarity

* rename send -> sendPromise to pair better with sendStream

* use retryIntervals instead of juggling MAX_REQUEST_RETRIES

- ensure debug messages are consistent between request streams +
promises
- set static constants

* DRY up status check + network failure + retry logic

- keeps the debug logic identical between promises + streams
- ensures all logic paths are also consistent
- consolidates the pop’ing of intervals in a single place

* find / replace fail

* derp

* make the logic actually correct, set intervals as cloned defaults for recursive lookup

* pass arg correctly

* reduce debugging noise, formatting

* rename intervals -> delaysRemaining for clarity

* added unit tests around getDelayForRetry

* set retryIntervals as default options correctly, add unit tests


Co-authored-by: Brian Mann <brian.mann86@gmail.com>
Co-authored-by: Jennifer Shehane <jennifer@cypress.io>
Co-authored-by: Gleb Bahmutov <gleb.bahmutov@gmail.com>
2019-05-15 12:46:55 -04:00
Zach Bloomquist
4b060362e7 Fix automatically loading fixtures with more than one . in the name (#3606)
Fixes #1402

Also fixes a few TODO comments in the fixtures.coffee file along the way
2019-03-06 10:15:05 -05:00
Jennifer Shehane
0a87734bd5 Update error message when record key not found for project (#3628)
- close #1449
2019-03-01 16:57:12 -05:00
Zach Bloomquist
ef9b8f7b38 Launch browsers by path: --browser /path/to/browser (#3337)
* adding multiple possible binary names for linux

* windows launcher doesn't consider "binary", so don't pass it

* adding test for multiple binary names

* Stronger typing, clearer variable names

* Stronger typing, clearer variable names

* cleanup

* cleanup

* clean up type- why isn't this being linted?

* Add more aliases (#3217)

* launcher changes to use Browser throughout, also clarifying FoundBrowser/Browser distinction

* wip

* wip

* update tests to expect objects

* removing errant debugger calls

* Fixing tests

* desktop-gui: use displayName for display

* ' -> "

* launcher: add definitions for google chrome beta and unstable

* server: fallthrough to using chrome helper

* server: changes for run mode to pick correct version

* desktop-gui: add displayName to fixtures

* server: isolating bug with runmode

* browser was a string all along

* server: re-promisify browser detection

* launcher: remove chrome-beta for now, needs some more tweaking for that to work 100 percent

* launcher: cleaning up types

* launcher: fix type comflict when filtering browsers (#3258)

* launcher: cast Windows foundbrowsers

* launcher: mapSeries -> map

* launcher: clean up launcher, change 1 call in server to match

* launcher: test that browsers contains what we like it to

* whoops

* server: accept path in runmode

* launcher: changes for detectByPath [wip]

* server: update tests to use new errors

* launcher: error message cleanup

* launcher: detectByPath working with CLI client

* launcher: detectByPath tests

* launcher: cli client for detectByPath

* server: update error msg snapshot

* cli: allow passing --browser to open mode

* server: using --browser=/path/ works in run and open mode!!

* launcher: change displayName of custom browsers

* server: find browser with highest version property by default

* launcher: update tests, clean up types

* server: fix tests

* server: fix tests

* cli: update help snapshots

* launcher: tests

* server: wip

* server, launcher: clean up errors

* server: add unit tests for events

* server: change e2e helper to support custom browser strings in stdout

* server: e2e tests for browser by path

* server: if this break that

* server: clean up and fix? tests

* decoffeeate, entypescriptify

* server: fix test

* cli: fix whitespace

* cli: remove external browser notice

* server: detect a browser to use for the e2e launch-by-path test

* server: make stackTraceLinesRe not match all sentences with 'at' in them

* server, launcher: update 'not found at path' error msg

* server: clean up browser switch

* server: customBrowserPath

* server: update snapshots that were affected by the old stackLineRe

* server: update stubs

* server: update BROWSER_NOT_FOUND_BY_PATH to use error objects

* server: backticks in snapshots break snapshots

* server: forgetting to save without formatting will be my downfall

* server: remove comment

* desktop-gui: make custom browsers chosen

* desktop-gui, launcher: update tests
2019-02-16 15:12:17 -05:00
Lila Conlee
44a6d8bd76 Update desktop GUI errors (#2993)
- Fixes #2955
2019-01-22 16:29:09 -05:00
Chris Breiding
907bd772e9 Add warnings/errors for recording limits (#2397)
Closes #1689 

The error/warning messages need work and the properties used (daysLeft, limit, etc) need to be aligned with the API once implemented.
2018-11-06 15:40:02 -05:00
Chris Breiding
70b0fa21eb merge handlers for multiple task registrations (#2289) 2018-09-08 19:06:57 -04:00
Jennifer Shehane
4ead424d4c some minor edits and typo fixes to error messages (#2287)
- some minor edits and typo fixes to error messages
2018-08-06 20:15:20 -04:00
Brian Mann
d0c38e9e13 error handling API response failures + retries (#2281)
* display a nice error for handling API response failures + retries

* fixes failing tests
2018-08-06 08:36:10 -04:00
Amir Rustamzadeh
f313dd0b84 iterate through specs in parallel (#2154)
- fixes: #2153
- fixes: #1566
- fixes: #1690
- fixes: #2275
- fixes: #2276
2018-08-06 07:24:19 -04:00
Brian Mann
4053596a88 validate cli + env var config, add breaking config changes (#1784)
* fixes #1783 validate configuration values after they're set from CLI + Env Vars

* fixes #1754 remove screenshotOnHeadlessFailure and rename trashAssetsBeforeHeadlessRuns

* derp copypasta fail
2018-05-28 01:27:54 -04:00
Brian Mann
fc423ca467 WIP creating a new banner + summary table for runs #1588 (#1722)
* WIP creating a new banner + summary table for runs [skip ci]

* WIP continuing to iterate on new run output, use tables for results [skip ci]

* log -> debug

* switch out 400 -> 412 status code on schema errors

* update humanTime to take short or long formats

* debug the starting argv

* create terminal-size wrapper to return terminal columns + rows

* created custom terminal tables for displaying new stdout results for running multiple specs

- made tables responsive when columns < 100

* add e2e tests around exiting with error when browser cannot be found or specs cannot be found

- instead of creating a run, immediately die without making any requests
- improve BROWSER_NOT_FOUND error message

* strip ansi color codes from terminal spec, use chalk instead of colors

* pass webUrl from recording runs to be able to display URL before tests run

* finished new stdout flow for running specs iteratively

* when webUrl is present, display the recorded run in the header + footer

- move from ‘white’ to ‘reset’
- tighten up intro table

* change results output from "Video Captured" to "Video" to pair with newest config option

* use runUrl, not webUrl, fix white space, ensure that e2e snapshots are accurately normalized

* use console.log, not console.error to capture stdout for dashboard

* e2e tests around logging warnings when cannot record video

* 0 seconds -> X seconds

* Tests Starting -> Run Starting, Tests Finished -> Results

* Started video recording -> hr line break

* All Done -> Run Finished

* evenly space stdout section headers

* add a new line when exiting early from bundle errors

* sanitize stdout better, replace duration seconds, etc

* update all the various e2e stdout snapshots

* ensure that COLUMNS env is set to 100 for snapshots to pass

* better logging for env var

* env -> stdout

* store as string

* remove columns

* echo columns

* add columns back in

* comment out

* set terminal columns

* cleanup circle.yml [skip ci]

* normalize more result properties for browser, os, cypress

* set columns for server tests

* enforce no colors and correct columns + rows for e2e tests, fix cloning env vars

* update e2e tests for newest stdout

* set env vars for columns + lines for e2e tests, remove stty

* fix snapshots when duration is X second, X seconds, or XX seconds

* use process.env.COLUMNS not process.stdout

* ugh

* stringify

* log crap out

* quotes oh my

* fix failing e2e snapshot tests

* fix failing e2e tests

* fix failing integration tests

* fixes failing unit tests
2018-05-21 18:37:35 -04:00
Gleb Bahmutov
9bed00a328 Do not fail if fork pr does not have record key 1193 (#1505)
* server: do not fail if missing record key from fork PR, close #1193

* remove .only

* fix white space

* add e2e tests around forked PR's warning but running without recording
2018-05-16 04:50:28 -04:00
Brian Mann
4b368830fa send array of specs to API + platform (#1682)
* WIP: refactor runs and recordings, update to new API updates, iteratively send spec results [skip ci]

* update / add schema deps

* add takenAt to screenshots payload

* WIP: refactor recordings to send correctly payloads, iterative through each spec, error handling, e2e tests

* add missing properties, remove hacks, upload stdout, passing tests

* normalize wall clock for newest schema spec

* rename projectPath -> projectRoot for clarity

* normalize specPattern to be relative to projectRoot

* comment out lib/api debug code

* WIP fixes a lot of failing tests

* many more tests around recording

* WIP: update to use x-os-name on all instead of platform

* WIP: update to route version 4 for creating instances

* server: upgrade json-schemas to 4.7.2

* remove debug logs

* fix stdout not being restored correctly between specs

* test all the edge cases with failed api interactions and early exits

* add e2e tests around recording without projectID

* add e2e tests around recording without recordKey

* refactored all tests surrounding record mode, removed duplicates, tested only edge cases

* fixes #1692

* fix failing unit tests, bump schemas

* bump sinon, replace custom sandbox

* fix sinon@5 not restoring fake timers automatically

* missing e2e record snapshots

* fix failing tests, don't pass config through each run, remove old timings in favor of spec isolation

* more e2e test fixes

* add e2e tests around uploading artifacts, fix bug with not uploading videos when it should

- cleanup some dead code
- add debug logs

* cleanup dead code, remove notion of failingTests
2018-05-13 20:24:46 -04:00
Brian Mann
9efad08f8c fixes #1585 exit with status 1 whenever no specs were found (#1587)
* fixes #1585 exit with status 1 whenever no specs were found

* add a couple tests around spec video file name

* cleanup old circleci branch filters

* bump up range of expected timings

* rename specsUtil for clarity, fix failing tests
2018-04-15 00:43:34 -04:00
Brian Mann
70e5916a28 server: fixes #1192 pass reporter errors along and provide stack if module doesn't exist (#1411) 2018-03-04 01:17:06 -05:00
Brian Mann
c1bcb2f641 Issue 1042 (#1057)
* server: remove unused file

* server: WIP, start implementing accepting plugin returning promise and yielding config overrides [skip ci]

* server: cleanup the env + environentVariables disaster. simplify and only use 'env'

* server, desktop-gum: rename 'env' to 'cypressEnv' to avoid conflicts, fix failing tests

* server: modify plugins error content to be clearer

* runner, driver: more environmentVariables -> env cleanup

* fixes #509 return the complete configuration object to Cypress.config()

* fixes #1042 enable plugins to return a promise and modify config

* desktop-gui: add 'plugin' override to configuration display

* server: bug fix when plugin cause a project not to open

* desktop-gui: fix for failing e2e test

* server: make errors clearer when plugins crash

* server: fix bug with PLUGINS_FUNCTION_ERROR not sending right arguments

- fix failing tests
- improve plugin error content

* server: fix failing snapshots, preprocessor is invoked for support + spec files

* server: fix tests, don't watch the support file initially

- this was causing a problem where unhandled preprocessor errors were
causing the entire process to hang.
- this was happening because we weren’t properly running support files
through a custom preprocessor initially

* fixes failing tests
2017-12-11 20:53:23 -05:00
Chris Breiding
1ec6f7815f Create plugins API and add preprocessor plugin support (#888)
* fix broken menu specs

* extract browserify code and create preprocessor abstraction

* server: remove unused code

* server: implement logic for importing preprocessor

* server: add eslintrc and ignore test fixtures

* server: implement plugins API

* server: add node_cache module to manage clearing a package and its dependencies

* server: clear plugins file from cache to re-require it

* server: set pluginsFile to absolute path

* server: add debug logging to plugins

* server: watch pluginsFile

* server: throw the right error when requiring plugins file

* server: organize config folders and keys visually

* server: scaffold plugins file

* server: require plugins file via nodeCache.require

* server: update browserify preprocessor name and point to github for now

* server: remove redundant integration tests

* server: always scaffold plugins file, except when falsey

* server: add plugins file test to scaffold spec

* server: always provide util.fileUpdated to preprocessor

* server: update browserify-preprocessor dep

* server: fix integration tests

* server: remove redundant logic

* server: catch errors thrown synchronously by plugins function

* server: reduce chances of flakyness in async test

* driver: fix broken spec

* server: don’t check for dirname of supportFile and pluginFile when we know they’re already a directory

* driver: there is no dist

* server: update default options sent to browserify preprocessor

* server: send config into browserify instead of options

* server: add test around default preprocessor

* server: update browserify preprocessor

* server: refactor events spec to better handle multiple calls to handleEvent

* server: add tests around project:open events

* server/desktop-gui: handle errors when plugins file changes

* server: fix events spec

* server: allow any file types to be spec files

* server: validate config.pluginsFile

* server: remove link to error doc

* server: improve plugins error message

* server: simplify example plugins file

* server: on:spec:file:preprocessor -> file:preprocessor

* server: wrap plugins in a child process

* server: rename specsGlob to testFiles

* server: bump browserify-preprocessor ‘version’

* server: fix references left by merge

* server: fix incorrect function names

* server: fix plugin error serialization

* server: display error if plugins process has an uncaught exception or unhandled rejection

* server: reset domain on reset

* server: exit early on plugin error headlessly

* server: add e2e tests for plugins/preprocessors

* server: remove obselete node cache file

* server: fix integration tests

* server: log plugins error before exiting

* server: use npm version of browserify preprocessor

* server: extract string-splitting function

* socket: use path.join in spec

* server: eslint ignore scaffold files

* server: fix scaffolded plugins file links, add snapshot test

* server: if app_spec runs, cause a failure [skip ci]

- tested locally
2017-11-19 23:20:54 -05:00
Chris Breiding
43c959696f desktop-gui: show message on runs tab when unable to connect to api server 2017-10-06 14:45:34 -04:00
Chris Breiding
ee49d1cdfa desktop-gui: improve login on runs tab (wip) 2017-09-29 17:28:52 -04:00
Brian Mann
ea451d48cd server: exit when running headed, add better warning when running electron headed not recording video 2017-09-17 21:38:53 -04:00
Brian Mann
8d0dd8fb3e server: fixes #658, improve the error when reporter isn't found and enable absolute path to reporters 2017-09-16 16:56:43 -04:00
Brian Mann
a82cac7c70 server: improve error message, add blank cypress.json files for e2e fixtures 2017-09-06 05:38:56 -04:00