Files
cypress/DEPLOY.md
Andrew Smith 0a6a2abcec Replace npm with yarn (#5555)
* enable using yarn

* enable lerna and yarn workspaces

No longer can reliably access node_modules via thei node_modules directory; yarn may optimize it via hoisting it up. This meant updating JS tasks that were copying files directly from node_modules directories. In these cases, pulled in a new package to resolve these correctly.

SCSS files remain impacted, but cannot easily import via JS. These paths have been modified, but it feels dangerous and incorrect to reach into a node_module to grab files like this.

Many prebuild steps were removed. I **think** the purpose of `check-deps-pre` is no longer needed, but need to confirm this.

* enabling test-unit script

removal of pretest-unit due to check-deps-pre

* removal of all  `check-deps` and `check-deps-pre`

I do not think these are needed anymore

* remove npm run all

Replaced by `lerna run` and `lerna run --scope`

* use yarn in circle CI

appveyor to soon follow

* yarn in appveyor

* remove need for bin-upplaces using bin-up have a dev dependency on some package; mostly mocha. This mocha package will be optimized by yarn workspaces by hoisting it into the root; which is effectively what bin-up usage was mimicing.

* replace npm run with yarn in package files

* replace explict paths to internal packages with yarn bin

* remove unecessary link packages script and references

* properly require package for mocha

* removing yarn test in this package as it was just a proxy for test-*

* yarn test in root now reflects what is run in circle ci

* relax yarn version requirement for circle ci

* @packages/cli is really just cypress

also run test in parallel via `yarn test`; it runs test in all 10 packages

* ensure postinstall is called before prebuild

yarn does not call postinstall when nothing has installed; such is the case when everything has been cached

* vscode config uses yarn

* cannot rely on which node_modules dir a package is in

find the right node_module dir via `resolve-pkg`

* further relax yarn version for OSX build in circle ci

* resolve failing test

Regarding direct access to node_modules

* node version in mac build not sufficient

ignoring check of node/yarn versions when installing

* preinstall script does not exist

I **think** the reason to not install packages' modules (the reason for `--ignore-scripts` may be irrelevant with yarn workspaces managing the packages)

* upgrade mocha that was using --file

`--file` was introduce in >=5

* scope test runs to correct packages

* explictly close connections; otherwise script hangs after tests run

* refactor so npm and npx commads can be created

npx is used to utlize lerna for running across all packages sans cli

* properly resolve socket.io-client node_module

We cannot reach directly into node_modules to grab the socket.io-client file. Furthermore, running it locally will require resolving from a different cwd, the repo root, as opposed to the package's dist directory.

The existence of the file deterimes whether we are building the binary or running locally.

* remove unnecessary arg in vscode debug config

enable debugging in @packages/server to troubleshoot failing test

* another npm command found to be changed to yarn

* properly print colors to terminal

`lerna run` does not appear to print colors to the terminal properly. Changed to leverage `lerna exec`

See https://github.com/lerna/lerna/issues/1168

* add missing test scripts to packages that have partially have them

'test', 'test-unit', 'test-watch', and 'test-debug'

This makes running them from the root easier and enables debugging via IDE on more packages

* properly patch package in yarn workspaces

* another package needing to postinstall

* use existing script over explicit lerna exec

* patches must be copied to dist

* return to building on postinstall

* do not hoist driver's packagesThere are too many places we need driver's packages to not be hoisted when testing. We have *.html files in test with script tags srcing node_modules.

* wait-on is used in circle-ci and needs to be a root dep

No longer hoisted by `@packages/driver`. This previously worked only because it was hoisted. Adding it to root properly.

* update documentation based on new ergonomics

* launcher has unit tests that should be runnable via top level task

* more concise way to run build within cypress scope

* fix for unit test that never seemed to work

`getPathToExample` is really `getPathToExamples`

* bust cache based on yarn.lock changing

* define intra-dependencies among packages

Due to the way we build the binary, all are considered dev dependencies.

* address `jquery.scrollto` mismatched jquery version

Tell yarn to explicitly resolve `jquery@3.1.1` for any dependencies and sub-dependencies. The root issue is that `jquery.scrollto` package specifies jquery as a dependency instead of a peer dependency (which is correct). Its jquery version is set to the `>=1.8` which then resolves to 3.4.1. In doing so, it will patch its jquery instead of ours; meaning that `$.scrollTo` is not defined.

* add/remove deps from renovate

- bin-up is gone in favor of yarn workspaces and lerna from root
- check-deps is gone in favor of `yarn check --integrity`
- lerna added

* few misc yarn lock updates

* hope that this may speed up build binary

installing several of the same dep in >1 package will likely be sped up using yarn due to its machine-wide caching.

* yarn pack prefix version with a `v`

* auto-run `yarn install` when deps become out of date

When switching to a branch with a different set of required deps, running any of the main top level tasks (as seen in the `./CONTRIBUTING.md` guide) will check for deps changes via `yarn check --integrity` and run `yarn` in root if deps are out of sync.

* add clean top level task, remove prebuild

install => build => prebuild => check => install === no good

* address code review to undo formatting changes

* favor default imports over deconstructing imports

* favor @package resolution; pr feedback

* include a specific revision

for some reason, it cannot resolve the original SHA `29dafed297142d3b8a9d8a01842cbdf249a98b72`

using the next closes SHA

* properly cache yarn packagesfollowed guide: https://circleci.com/docs/2.0/yarn/

* lower barrier for contributors to get started

relax node version to be 12.0.0 or higher; will use node version found in `.node-version` in CI

* updates required after merging latest development

was getting type errors when building. recreating the `yarn.lock` file seemed to resolve the issue.

* hidden dep on a specific type version in server

made dep explicit and now it can build.

* update request to patched version

non-breaking changes to request was updated in server, but not everywhere else. It appears that transitive deps were using the 2.88.0 request version instead of the updated one for the server package.

* missing commit from previous commit/merge

* do not force a higher version of yarn than the default on circleci

* exclude e2e.js helper from stop-only command

* trying to bust node_modules cache.https://github.com/yarnpkg/yarn/issues/6412#issuecomment-537787740

* incorrect quotation tick placement

* fix: properly postinstall parse-domain

Do not explicitly script the shell invocation of a dependency's postinstall (parse-domain). The dependency's node_modules may not be where the package thinks they are and are not properly resolved.

It appears that postinstall will automatically be called when installing with yarn.

* run all CI stages

* fix win-appveyor-build.js

* Update CONTRIBUTING.md

Co-Authored-By: Zach Bloomquist <github@chary.us>

* Update CONTRIBUTING.md

Co-Authored-By: Zach Bloomquist <github@chary.us>

* Update CONTRIBUTING.md

Co-Authored-By: Zach Bloomquist <github@chary.us>

* Server package watching a specific test works correctly.

* PR feedback

* Correctly use yarn to pack (to get package size)

* correct size calculation

`yarn pack` does not output the file name like `npm pack` does. Correct this by explictly setting the filename.

* PR feedback

* PR feedback

* update readmes and testing commands from them

* do not include this in default build

* yarn is now installed by default

https://github.com/appveyor/ci/issues/1852

* do not include server in `yarn` auto build

* a few more npm references changed to yarn

* Revert "run all CI stages"

This reverts commit 9256aed99f.

* Update circle.yml

Co-Authored-By: Zach Bloomquist <github@chary.us>

Co-authored-by: Zach Bloomquist <github@chary.us>
2020-02-11 12:40:07 +06:30

11 KiB
Raw Blame History

Deployment

Anyone can build the binary and NPM package, but you can only deploy the Cypress application and publish the NPM module cypress if you are a member of the cypress NPM organization.

See the publishing section for how to build, test and publish a new official version of the binary and cypress NPM package.

Set next version on CIs

We build the NPM package and binary on all major platforms (Linux, Mac, Windows) on different CI providers. In order to set the version while building we have to set the environment variable with the new version on each CI provider before starting the build.

Use script command yarn set-next-ci-version to do this.

Building

Building the NPM package

⚠️ Note: The steps in this section are automated in CI, and you should not generally need to do them yourself.

Building a new NPM package is very quick.

  • Increment the version in the root package.json
  • yarn build --scope cypress

The steps above:

  • Build the cypress NPM package
  • Transpile the code into ES5 to be compatible with the common Node versions
  • Put the result into the cli/build folder.

You could publish from there, but first you need to build and upload the binary with the same version; this guarantees that when users do npm i cypress@<x.y.z> they can download the binary with the same version x.y.z from Cypress's CDN service.

Building the binary

⚠️ Note: The steps in this section are automated in CI, and you should not generally need to do them yourself.

First, you need to build, zip and upload the application binary to the Cypress server.

You can use a single command to do all tasks at once:

yarn binary-deploy

Or you can specify each command separately:

yarn binary-build
yarn binary-zip
yarn binary-upload

You can pass options to each command to avoid answering questions, for example

yarn binary-deploy --platform darwin --version 0.20.0
yarn binary-upload --platform darwin --version 0.20.0 --zip cypress.zip

If something goes wrong, see the debug messages using the DEBUG=cypress:binary ... environment variable.

Because we had many problems reliably zipping the built binary, for now we need to build both the Mac and Linux binary from Mac (Linux binary is built using a Docker container), then zip it from Mac, then upload it.

Building Linux binary in Docker

If you are using a Mac you can build the linux binary if you have docker installed.

yarn binary-build-linux

Publishing

Before Publishing a New Version

In order to publish a new cypress package to the NPM registry, we must build and test it across multiple platforms and test projects. This makes publishing directly into the NPM registry impossible. Instead, we have CI set up to do the following on every commit to develop:

  1. Build the NPM package with the new target version baked in.
  2. Build the Linux/Mac binaries on CircleCI and build Windows on AppVeyor.
  3. Upload the binaries and the new NPM package to cdn.cypress.io under the "beta" folder.
  4. Launch the test projects like cypress-test-node-versions and cypress-test-example-repos using the newly-uploaded package & binary instead of installing from the NPM registry. That installation looks like this:
    export CYPRESS_INSTALL_BINARY=https://cdn.../binary/<new version>/<commit hash>/cypress.zip
    npm i https://cdn.../npm/<new version>/<commit hash>/cypress.tgz
    

Multiple test projects are launched for each target operating system and the results are reported back to GitHub using status checks so that you can see if a change has broken real-world usage of Cypress. You can see the progress of the test projects by opening the status checks on GitHub:

Screenshot of status checks

Once the develop branch for all test projects are reliably passing with the new changes, publishing can proceed.

Steps to Publish a New Version

In the following instructions, "X.Y.Z" is used to denote the version of Cypress being published.

  1. Make sure that if there is a new cypress-example-kitchensink version, the corresponding dependency in packages/example has been updated to that new version.

  2. Make sure that you have the correct environment variables set up before proceeding.

    • You'll need Cypress AWS access keys in aws_credentials_json, which looks like this:
      aws_credentials_json={"bucket":"cdn.cypress.io","folder":"desktop","key":"...","secret":"..."}
      
    • You'll need a GitHub token, a CircleCI token, and a cypress-io account-specific AppVeyor token in ci_json:
      ci_json={"githubToken":"...","circleToken":"...","appVeyorToken":"..."}
      
    • Tip: Use as-a to manage environment variables for different situations.
  3. Use the move-binaries script to move the binaries for <commit sha> from beta to the desktop folder for <new target version>

    yarn move-binaries --sha <commit sha> --version <new target version>
    
  4. Publish the new NPM package under the dev tag. The unique link to the package file cypress.tgz is the one already tested above. You can publish to the NPM registry straight from the URL:

    yarn publish https://cdn.../npm/X.Y.Z/<long sha>/cypress.tgz --tag dev
    
  5. Double-check that the new version has been published under the dev tag using npm info cypress or available-versions. Example output:

    dist-tags:
    dev: 3.4.0     latest: 3.3.2
    
  6. Test cypress@X.Y.Z again to make sure everything is working. You can trigger test projects from the command line (if you have the appropriate permissions)

    node scripts/test-other-projects.js --npm cypress@X.Y.Z --binary X.Y.Z
    
  7. Test the new version of Cypress against the Cypress dashboard repo.

  8. Update and publish the changelog and any release-specific documentation changes in cypress-documentation.

  9. Make the new NPM version the "latest" version by updating the dist-tag latest to point to the new version:

    yarn dist-tag add cypress@X.Y.Z
    
  10. Run binary-release to update the download server's manifest, set the next CI version, and create an empty version commit:

    yarn run binary-release --version X.Y.Z --commit
    
  11. If needed, push out any updated changes to the links manifest to on.cypress.io.

  12. If needed, deploy the updated cypress-example-kitchensink to example.cypress.io by following these instructions under "Deployment".

  13. Update the releases in ZenHub:

    • Close the current release in ZenHub.
    • Create a new patch release (and a new minor release, if this is a minor release) in ZenHub, and schedule them both to be completed 2 weeks from the current date.
    • Move all issues that are still open from the current release to the appropriate future release.
  14. Bump version in package.json and commit it to develop using a commit message like release X.Y.Z [skip ci]

  15. Tag this commit with vX.Y.Z and push that tag up.

  16. Merge develop into master and push that branch up.

  17. Inside of cypress-io/release-automations:

    • Publish GitHub release to cypress-io/cypress/releases using package set-releases:
      cd set-releases && npm run release-log -- --version X.Y.Z
      
    • Add a comment to each GH issue that has been resolved with the new published version using package issues-in-release:
      cd issues-in-release && npm run do:comment -- --release X.Y.Z
      
  18. Publish a new docker image in cypress-docker-images under included for the new cypress version.

  19. Decide on the next version that we will work on. For example, if we have just released 3.7.0 we probably will work on 3.7.1 next. Set it on CI machines.

  20. Try updating as many example projects to the new version. You probably want to update by using Renovate dependency issue like cypress-example-todomvc "Update Dependencies (Renovate Bot). Try updating at least the following projects:

  21. Check if any test or example repositories have a branch for testing the features or fixes from the newly published version x.y.z. The branch should also be named x.y.z. Check all cypress-test-* and cypress-example-* repositories, and if there is a branch named x.y.z, merge it into master.

    Test Repos

    Example Repos

Take a break, you deserve it! 😎