diff --git a/README.md b/README.md index 53363df..6668efe 100644 --- a/README.md +++ b/README.md @@ -1,24 +1,21 @@ # @sparticuz/chromium [![@sparticuz/chromium](https://img.shields.io/npm/v/@sparticuz/chromium.svg?style=for-the-badge)](https://www.npmjs.com/package/@sparticuz/chromium) -[![TypeScript](https://img.shields.io/npm/types/chrome-aws-lambda?style=for-the-badge)](https://www.typescriptlang.org/dt/search?search=chromium) -[![Chromium](https://img.shields.io/badge/chromium-51_MB-brightgreen.svg?style=for-the-badge)](bin/) +[![Chromium](https://img.shields.io/github/size/sparticuz/chromium/bin/chromium.br?label=Chromium&style=for-the-badge)](bin/) +[![npm](https://img.shields.io/npm/dw/@sparticuz/chromium?label=%40sparticuz%2Fchromium&style=for-the-badge)](https://www.npmjs.com/package/@sparticuz/chromium) +[![npm](https://img.shields.io/npm/dw/@sparticuz/chromium-min?label=%40sparticuz%2Fchromium-min&style=for-the-badge)](https://www.npmjs.com/package/@sparticuz/chromium-min) [![Donate](https://img.shields.io/badge/donate-paypal-orange.svg?style=for-the-badge)](https://paypal.me/sparticuz) ## Chromium for Serverless platforms -This package was originally forked from [alixaxel/chrome-aws-lambda#264](https://github.com/alixaxel/chrome-aws-lambda/pull/264). -The biggest difference, besides the chromium version, is the inclusion of some code from https://github.com/alixaxel/lambdafs, -as well as dropping that as a dependency. Due to some changes in WebGL, the files in bin/swiftshader.tar.br need to -be extracted to `/tmp` instead of `/tmp/swiftshader`. This necessitated changes in lambdafs. +[sparticuz/chrome-aws-lambda](https://github.com/sparticuz/chrome-aws-lambda) was originally forked from [alixaxel/chrome-aws-lambda#264](https://github.com/alixaxel/chrome-aws-lambda/pull/264). +The biggest difference, besides the chromium version, is the inclusion of some code from https://github.com/alixaxel/lambdafs, as well as dropping that as a dependency. Due to some changes in WebGL, the files in bin/swiftshader.tar.br need to be extracted to `/tmp` instead of `/tmp/swiftshader`. This necessitated changes in lambdafs. -However, it quickly became difficult to maintain because of the pace of `puppeteer` updates. This package, `@sparticuz/chromium`, -is not chained to `puppeteer` versions, but also does not include the overrides and hooks that the original package contained. It is only `chromium`, as well as the special code needed to decompress the brotli package. +However, it quickly became difficult to maintain because of the pace of `puppeteer` updates. This package, `@sparticuz/chromium`, is not chained to `puppeteer` versions, but also does not include the overrides and hooks that the original package contained. It is only `chromium`, as well as the special code needed to decompress the brotli package, and a set of predefined arguments tailored to serverless usage. ## Install -[`puppeteer` ships with a prefered version of `chromium`](https://pptr.dev/faq/#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy). -In order to figure out what version of `@sparticuz/chromium` you will need, please visit [Puppeteer's Chromium Support page](https://pptr.dev/chromium-support). +[`puppeteer` ships with a prefered version of `chromium`](https://pptr.dev/faq/#q-why-doesnt-puppeteer-vxxx-work-with-chromium-vyyy). In order to figure out what version of `@sparticuz/chromium` you will need, please visit [Puppeteer's Chromium Support page](https://pptr.dev/chromium-support). > For example, as of today, the latest version of `puppeteer` is `18.0.5`. The latest version of `chromium` stated on `puppeteer`'s support page is `106.0.5249.0`. So you need to install `@sparticuz/chromium@106`. @@ -30,6 +27,7 @@ npm install --save-dev @sparticuz/chromium@$CHROMIUM_VERSION ``` If your vendor does not allow large deploys (`chromium.br` is 50+ MB), you'll need to host the `chromium-v#-pack.tar` separatly and use the [`@sparticuz/chromium-min` package](https://github.com/Sparticuz/chromium#-min-package). + ```shell npm install --save @sparticuz/chromium-min@$CHROMIUM_VERSION ``` @@ -41,6 +39,8 @@ If you wish to install an older version of Chromium, take a look at [@sparticuz/ The @sparticuz/chromium version schema is as follows: `MajorChromiumVersion.MinorChromiumIncrement.@Sparticuz/chromiumPatchLevel` +Beacuse this package follows Chromium's releases, it does NOT follow semantic versioning. **Breaking changes can occur with the 'patch' level.** Please check the release notes for information on breaking changes. + ## Usage This package works with all the currently supported AWS Lambda Node.js runtimes out of the box. @@ -50,13 +50,23 @@ const test = require("node:test"); const puppeteer = require("puppeteer-core"); const chromium = require("@sparticuz/chromium"); +// Optional: If you'd like to use the legacy headless mode. "new" is the default. +chromium.setHeadlessMode = true; + +// Optional: If you'd like to disable webgl, true is the default. +chromium.setGraphicsMode = false; + +// Optional: Load any fonts you need. Open Sans is included by default in AWS Lambda instances +await chromium.font( + "https://raw.githack.com/googlei18n/noto-emoji/master/fonts/NotoColorEmoji.ttf" +); + test("Check the page title of example.com", async (t) => { const browser = await puppeteer.launch({ args: chromium.args, defaultViewport: chromium.defaultViewport, executablePath: await chromium.executablePath(), headless: chromium.headless, - ignoreHTTPSErrors: true, }); const page = await browser.newPage(); @@ -73,8 +83,8 @@ test("Check the page title of example.com", async (t) => { ```javascript const test = require("node:test"); // Need to rename playwright's chromium object to something else -const { chromium: playwright } = require('playwright-core'); -const chromium = require('@sparticuz/chromium'); +const { chromium: playwright } = require("playwright-core"); +const chromium = require("@sparticuz/chromium"); test("Check the page title of example.com", async (t) => { const browser = await playwright.launch({ @@ -92,18 +102,19 @@ test("Check the page title of example.com", async (t) => { assert.strictEqual(pageTitle, "Example Domain"); }); ``` -You should allocate at least 512 MB of RAM to your Lambda, however 1600 MB (or more) is recommended. + +You should allocate at least 512 MB of RAM to your instance, however 1600 MB (or more) is recommended. ### -min package -The -min package DOES NOT include the chromium brotli files. There are a few instances where this -is useful. Primarily, this is useful when you have file size limits. +The -min package DOES NOT include the chromium brotli files. There are a few instances where this is useful. Primarily, this is useful when your host has file size limits. To use the -min package please install the `@sparticuz/chromium-min` package. When using the -min package, you need to specify the location of the brotli files. In this example, /opt/chromium contains all the brotli files + ``` /opt /chromium @@ -111,22 +122,19 @@ In this example, /opt/chromium contains all the brotli files /chromium.br /swiftshader.tar.br ``` + ```javascript const browser = await puppeteer.launch({ args: chromium.args, defaultViewport: chromium.defaultViewport, executablePath: await chromium.executablePath("/opt/chromium"), headless: chromium.headless, - ignoreHTTPSErrors: true, }); ``` -In the following example, https://www.example.com/chromiumPack.tar contains all the brotli files. -Generally, this would be a location on S3, or another very fast downloadable location, -that is close to your function's execution location. -@sparticuz/chromium will download the pack tar file, untar the files to /tmp/chromium-pack, -then will un-brotli the files to /tmp/chromium. The next iteration will have /tmp/chromium exist -and will use the already downloaded files. +In the following example, https://www.example.com/chromiumPack.tar contains all the brotli files. Generally, this would be a location on S3, or another very fast downloadable location, that is in close proximity to your function's execution location. + +On the initial iteration, `@sparticuz/chromium` will download the pack tar file, untar the files to `/tmp/chromium-pack`, then will un-brotli the `chromium` binary to `/tmp/chromium`. The following iterations will see that `/tmp/chromium` exists and will use the already downloaded files. The latest chromium-pack.tar file will be on the latest [release](https://github.com/Sparticuz/chromium/releases). @@ -134,51 +142,66 @@ The latest chromium-pack.tar file will be on the latest [release](https://github const browser = await puppeteer.launch({ args: chromium.args, defaultViewport: chromium.defaultViewport, - executablePath: await chromium.executablePath("https://www.example.com/chromiumPack.tar"), + executablePath: await chromium.executablePath( + "https://www.example.com/chromiumPack.tar" + ), headless: chromium.headless, - ignoreHTTPSErrors: true, }); ``` ### Examples + Here are some example projects and help with other services + - [Production Dependency](https://github.com/Sparticuz/chromium/tree/master/examples/production-dependency) - [Serverless Framework with Lambda Layer](https://github.com/Sparticuz/chromium/tree/master/examples/serverless-with-lambda-layer) +- [Serverless Framework with Pre-existing Lambda Layer](https://github.com/Sparticuz/chromium/tree/master/examples/serverless-with-preexisting-lambda-layer) - [Chromium-min](https://github.com/Sparticuz/chromium/tree/master/examples/remote-min-binary) -- AWS SAM *TODO* +- AWS SAM _TODO_ - [Webpack](https://github.com/Sparticuz/chromium/issues/24#issuecomment-1343196897) - [Netlify](https://github.com/Sparticuz/chromium/issues/24#issuecomment-1414107620) -### Running Locally +### Running Locally & Headless/Headful mode -This package will run in headless mode when `NODE_ENV = "test"`. If you want to run using your own local binary, set `IS_LOCAL` to anything. +This version of `chromium` is built using the `headless.gn` build variables, which does not appear to even include a GUI. [Also, at this point, AWS Lambda 2 does not support a modern version of `glibc`](https://github.com/aws/aws-lambda-base-images/issues/59), so this package does not include an ARM version yet, which means it will not work on any M Series Apple products. If you need to test your code using a headful or ARM version, please use your locally installed version of `chromium/chrome`, or you may use the `puppeteer` provided version. -## API +```shell +npx @puppeteer/browsers install chromium@latest --path /tmp/localChromium +``` -| Method / Property | Returns | Description | -| --------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `font(url)` | `{?Promise}` | Provisions a custom font and returns its basename. | -| `args` | `{!Array}` | Provides a list of recommended additional [Chromium flags](https://github.com/GoogleChrome/chrome-launcher/blob/master/docs/chrome-flags-for-tools.md). | -| `defaultViewport` | `{!Object}` | Returns more sensible default viewport settings. | -| `executablePath(location)` | `{?Promise}` | Returns the path the Chromium binary was extracted to. | -| `headless` | `{!boolean}` | Returns `true` if we are running on AWS Lambda or GCF. | +For more information on installing a specific version of `chromium`, checkout [@puppeteer/browsers](https://www.npmjs.com/package/@puppeteer/browsers). + +For example, you can set your code to use an ENV variable such as `IS_LOCAL`, then use if/else statments to direct puppeteer to the correct environment. + +```javascript +const browser = await puppeteer.launch({ + args: process.env.IS_LOCAL ? puppeteer.defaultArgs() : chromium.args, + defaultViewport: chromium.defaultViewport, + executablePath: process.env.IS_LOCAL + ? "/tmp/localChromium/chromium/linux-1122391/chrome-linux/chrome" + : await chromium.executablePath(), + headless: process.env.IS_LOCAL ? false : chromium.headless, +}); +``` ## Fonts -The Amazon Linux 2 AWS Lambda runtime is no longer provisioned with any font faces. +The Amazon Linux 2 AWS Lambda runtime is not provisioned with any font faces. Because of this, this package ships with [Open Sans](https://fonts.google.com/specimen/Open+Sans), which supports the following scripts: -* Latin -* Greek -* Cyrillic +- Latin +- Greek +- Cyrillic To provision additional fonts, simply call the `font()` method with an absolute path or URL: ```typescript -await chromium.font('/var/task/fonts/NotoColorEmoji.ttf'); +await chromium.font("/var/task/fonts/NotoColorEmoji.ttf"); // or -await chromium.font('https://raw.githack.com/googlei18n/noto-emoji/master/fonts/NotoColorEmoji.ttf'); +await chromium.font( + "https://raw.githack.com/googlei18n/noto-emoji/master/fonts/NotoColorEmoji.ttf" +); ``` > `Noto Color Emoji` (or similar) is needed if you want to [render emojis](https://getemoji.com/). @@ -187,8 +210,6 @@ await chromium.font('https://raw.githack.com/googlei18n/noto-emoji/master/fonts/ This method should be invoked _before_ launching Chromium. -> On non-serverless environments, the `font()` method is a no-op to avoid polluting the user space. - --- Alternatively, it's also possible to provision fonts via AWS Lambda Layers. @@ -206,6 +227,24 @@ Afterwards, you just need to ZIP the directory and upload it as a AWS Lambda Lay ```shell zip -9 --filesync --move --recurse-paths .fonts.zip .fonts/ ``` + +## Graphics + +By default, this package uses `swiftshader`/`angle` to do CPU acceleration for WebGL. This is the only known way to enable WebGL on a serverless platform. You can disable WebGL by setting `chromium.setGraphiceMode = false;` _before_ launching Chromium. Disabling this will also skip the extract of the `bin/swiftshader.tar.br` file, which saves about a second of initial execution time. Disabling graphics is recommended if you know you are not using any WebGL. + +## API + +| Method / Property | Returns | Description | +| ----------------------------------- | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `font(url)` | `Promise` | Provisions a custom font and returns its basename. | +| `args` | `Array` | Provides a list of recommended additional [Chromium flags](https://github.com/GoogleChrome/chrome-launcher/blob/master/docs/chrome-flags-for-tools.md). | +| `defaultViewport` | `Object` | Returns a sensible default viewport for serverless. | +| `executablePath(location?: string)` | `Promise` | Returns the path the Chromium binary was extracted to. | +| `setHeadlessMode` | `void` | Sets the headless mode to either `true` or `"new"` | +| `headless` | `true \| "new"` | Returns `true` or `"new"` depending on what version of chrome's headless you are running | +| `setGraphicsMode` | `void` | Sets the graphics mode to either `true` or `false` | +| `graphics` | `boolean` | Returns a boolean depending on whether webgl is enabled or disabled | + ## Compiling To compile your own version of Chromium check the [Ansible playbook instructions](_/ansible). @@ -223,6 +262,7 @@ make chromium.zip ``` The above will create a `chromium.zip` file, which can be uploaded to your Layers console. You can and should upload using the `aws cli`. (Replace the variables with your own values) + ```shell bucketName="chromiumUploadBucket" && \ versionNumber="107" && \ @@ -240,6 +280,7 @@ According to our benchmarks, it's 40% to 50% faster than using the off-the-shelf - Add the import or require for `puppeteer-core` - Change the browser launch to use the native `puppeteer.launch()` function - Change the `executablePath` to be a function. + ```diff -const chromium = require('@sparticuz/chrome-aws-lambda'); +const chromium = require("@sparticuz/chromium"); @@ -276,6 +317,7 @@ exports.handler = async (event, context, callback) => { return callback(null, result); }; ``` + ## Compression The Chromium binary is compressed using the Brotli algorithm. diff --git a/examples/serverless-with-preexisting-lambda-layer/README.md b/examples/serverless-with-preexisting-lambda-layer/README.md new file mode 100644 index 0000000..d6f780d --- /dev/null +++ b/examples/serverless-with-preexisting-lambda-layer/README.md @@ -0,0 +1,40 @@ +## Upload Lambda layer to AWS + +1. Download the layer zip from Github Releases + ![Step 1](docs/step1.png) +2. Create a S3 bucket, or use a pre-existing bucket, upload the zip, and copy the URL + ![Step 2](docs/step2.png) +3. Create a new layer or use a pre-existing layer. (If using a pre-existing layer, Create a new verion) + ![Step 3](docs/step3.png) +4. Use the S3 file to load into to AWS Lambda Layers + ![Step 4](docs/step4.png) +5. Add the layer to your serverless function + +```yaml +service: sls-with-preexisting-layer + +provider: + name: aws + runtime: nodejs18.x + stage: dev + region: us-east-1 + timeout: 300 + +functions: + chromium-test: + handler: index.handler + layers: + - arn:aws:lambda:us-east-1:************:layer:chromium:* +``` + +# BONUS + +These steps can easily be automated using the following code: + +```shell +$ chromiumVersion="112.0.0" +$ bucketName="chromiumUploadBucket" +$ wget "https://github.com/Sparticuz/chromium/releases/download/v${chromiumVersion}/chromium-v${chromiumVersion}-layer.zip" +$ aws s3 cp "chromium-v${chromiumVersion}-layer.zip" "s3://${bucketName}/chromiumLayers/chromium-v${chromiumVersion}-layer.zip" +$ aws lambda publish-layer-version --layer-name chromium --description "Chromium v${chromiumVersion}" --content "S3Bucket=${bucketName},S3Key=chromiumLayers/chromium-v${chromiumVersion}-layer.zip" --compatible-runtimes nodejs --compatible-architectures x86_64 +``` diff --git a/examples/serverless-with-preexisting-lambda-layer/docs/step1.png b/examples/serverless-with-preexisting-lambda-layer/docs/step1.png new file mode 100644 index 0000000..f52711d Binary files /dev/null and b/examples/serverless-with-preexisting-lambda-layer/docs/step1.png differ diff --git a/examples/serverless-with-preexisting-lambda-layer/docs/step2.png b/examples/serverless-with-preexisting-lambda-layer/docs/step2.png new file mode 100644 index 0000000..cfa9925 Binary files /dev/null and b/examples/serverless-with-preexisting-lambda-layer/docs/step2.png differ diff --git a/examples/serverless-with-preexisting-lambda-layer/docs/step3.png b/examples/serverless-with-preexisting-lambda-layer/docs/step3.png new file mode 100644 index 0000000..e2101b3 Binary files /dev/null and b/examples/serverless-with-preexisting-lambda-layer/docs/step3.png differ diff --git a/examples/serverless-with-preexisting-lambda-layer/docs/step4.png b/examples/serverless-with-preexisting-lambda-layer/docs/step4.png new file mode 100644 index 0000000..28827c9 Binary files /dev/null and b/examples/serverless-with-preexisting-lambda-layer/docs/step4.png differ diff --git a/examples/serverless-with-preexisting-lambda-layer/index.js b/examples/serverless-with-preexisting-lambda-layer/index.js new file mode 100644 index 0000000..039cdec --- /dev/null +++ b/examples/serverless-with-preexisting-lambda-layer/index.js @@ -0,0 +1,29 @@ +const puppeteer = require("puppeteer-core"); +const chromium = require("@sparticuz/chromium"); + +module.exports = { + handler: async () => { + try { + const browser = await puppeteer.launch({ + args: chromium.args, + defaultViewport: chromium.defaultViewport, + executablePath: await chromium.executablePath(), + headless: chromium.headless, + ignoreHTTPSErrors: true, + }); + + const page = await browser.newPage(); + + await page.goto("https://www.example.com", { waitUntil: "networkidle0" }); + + console.log("Chromium:", await browser.version()); + console.log("Page Title:", await page.title()); + + await page.close(); + + await browser.close(); + } catch (error) { + throw new Error(error.message); + } + }, +}; diff --git a/examples/serverless-with-preexisting-lambda-layer/package.json b/examples/serverless-with-preexisting-lambda-layer/package.json new file mode 100644 index 0000000..b3d109f --- /dev/null +++ b/examples/serverless-with-preexisting-lambda-layer/package.json @@ -0,0 +1,21 @@ +{ + "name": "serverless-with-lambda-layer", + "version": "0.0.0", + "description": "This package demonstrates using @sparticuz/chromium as a devDependency with a layer that contains the binaries", + "license": "ISC", + "author": { + "name": "Kyle McNally" + }, + "main": "index.js", + "scripts": { + "deploy": "sls deploy", + "test": "sls invoke --function chromium-test --log" + }, + "dependencies": { + "puppeteer-core": "19.6.3" + }, + "devDependencies": { + "@sparticuz/chromium": "110.0.0", + "serverless": "^3.27.0" + } +} diff --git a/examples/serverless-with-preexisting-lambda-layer/serverless.yml b/examples/serverless-with-preexisting-lambda-layer/serverless.yml new file mode 100644 index 0000000..503f54a --- /dev/null +++ b/examples/serverless-with-preexisting-lambda-layer/serverless.yml @@ -0,0 +1,14 @@ +service: sls-with-layer + +provider: + name: aws + runtime: nodejs18.x + stage: dev + region: us-east-1 + timeout: 300 + +functions: + chromium-test: + handler: index.handler + layers: + - arn:aws:lambda:us-east-1:************:layer:chromium:* diff --git a/source/helper.ts b/source/helper.ts index d6c87c1..5d1c673 100644 --- a/source/helper.ts +++ b/source/helper.ts @@ -17,6 +17,20 @@ export const isValidUrl = (input: string) => { } }; +/** + * Determines if the running instance is inside an AWS Lambda container. + * @returns + */ +export const isRunningInAwsLambda = () => { + if ( + process.env.AWS_EXECUTION_ENV && + /^AWS_Lambda_nodejs/.test(process.env.AWS_EXECUTION_ENV) === true + ) { + return true; + } + return false; +}; + export const downloadAndExtract = async (url: string) => new Promise((resolve, reject) => { const getOptions = parse(url) as FollowRedirOptions; diff --git a/source/index.ts b/source/index.ts index a1cc8c4..0c3739a 100644 --- a/source/index.ts +++ b/source/index.ts @@ -5,11 +5,11 @@ import { mkdirSync, symlink, } from "node:fs"; -import { IncomingMessage } from "node:http"; +import { https } from "follow-redirects"; import LambdaFS from "./lambdafs"; import { join } from "node:path"; import { URL } from "node:url"; -import { downloadAndExtract, isValidUrl } from "./helper"; +import { downloadAndExtract, isRunningInAwsLambda, isValidUrl } from "./helper"; /** Viewport taken from https://github.com/puppeteer/puppeteer/blob/main/docs/api/puppeteer.viewport.md */ interface Viewport { @@ -24,31 +24,27 @@ interface Viewport { /** * Specify device scale factor. * See {@link https://developer.mozilla.org/en-US/docs/Web/API/Window/devicePixelRatio | devicePixelRatio} for more info. - * @defaultValue 1 + * @default 1 */ deviceScaleFactor?: number; /** * Whether the `meta viewport` tag is taken into account. - * @defaultValue false + * @default false */ isMobile?: boolean; /** * Specifies if the viewport is in landscape mode. - * @defaultValue false + * @default false */ isLandscape?: boolean; /** * Specify if the viewport supports touch events. - * @defaultValue false + * @default false */ hasTouch?: boolean; } -if ( - process.env.AWS_EXECUTION_ENV !== undefined && - /^AWS_Lambda_nodejs(?:14|16|18)[.]x$/.test(process.env.AWS_EXECUTION_ENV) === - true -) { +if (isRunningInAwsLambda()) { if (process.env.FONTCONFIG_PATH === undefined) { process.env.FONTCONFIG_PATH = "/tmp/aws"; } @@ -64,16 +60,23 @@ if ( class Chromium { /** - * Downloads or symlinks a custom font and returns its basename, patching the environment so that Chromium can find it. - * If headless is not true, `null` is returned instead. + * Determines the headless mode that chromium will run at + * https://developer.chrome.com/articles/new-headless/#try-out-the-new-headless + * @values true or "new" */ - static font(input: string): Promise { - if (Chromium.headless !== true) { - return new Promise((resolve) => { - return resolve(null); - }); - } + private static headlessMode: true | "new" = "new"; + /** + * If true, the graphics stack and webgl is enabled, + * If false, webgl will be disabled. + * (If false, the swiftshader.tar.br file will also not extract) + */ + private static graphicsMode: boolean = true; + + /** + * Downloads or symlinks a custom font and returns its basename, patching the environment so that Chromium can find it. + */ + static font(input: string): Promise { if (process.env.HOME === undefined) { process.env.HOME = "/tmp"; } @@ -109,10 +112,7 @@ class Chromium { }); }); } else { - let handler = - url.protocol === "http:" ? require("http").get : require("https").get; - - handler(input, (response: IncomingMessage) => { + https.get(input, (response) => { if (response.statusCode !== 200) { return reject(`Unexpected status code: ${response.statusCode}.`); } @@ -142,47 +142,109 @@ class Chromium { * The canonical list of flags can be found on https://peter.sh/experiments/chromium-command-line-switches/. */ static get args(): string[] { - const result = [ - "--allow-running-insecure-content", // https://source.chromium.org/search?q=lang:cpp+symbol:kAllowRunningInsecureContent&ss=chromium - "--autoplay-policy=user-gesture-required", // https://source.chromium.org/search?q=lang:cpp+symbol:kAutoplayPolicy&ss=chromium + /** + * These are the default ares in puppeteer. + * https://github.com/puppeteer/puppeteer/blob/3a31070d054fa3cd8116ca31c578807ed8d6f987/packages/puppeteer-core/src/node/ChromeLauncher.ts#L185 + */ + const puppeteerFlags = [ + "--allow-pre-commit-input", + "--disable-background-networking", "--disable-background-timer-throttling", - "--disable-component-update", // https://source.chromium.org/search?q=lang:cpp+symbol:kDisableComponentUpdate&ss=chromium - "--disable-domain-reliability", // https://source.chromium.org/search?q=lang:cpp+symbol:kDisableDomainReliability&ss=chromium - "--disable-features=AudioServiceOutOfProcess,IsolateOrigins,site-per-process", // https://source.chromium.org/search?q=file:content_features.cc&ss=chromium - "--disable-ipc-flooding-protection", - "--disable-print-preview", // https://source.chromium.org/search?q=lang:cpp+symbol:kDisablePrintPreview&ss=chromium + "--disable-backgrounding-occluded-windows", + "--disable-breakpad", + "--disable-client-side-phishing-detection", + "--disable-component-extensions-with-background-pages", + "--disable-component-update", + "--disable-default-apps", "--disable-dev-shm-usage", - "--disable-setuid-sandbox", // https://source.chromium.org/search?q=lang:cpp+symbol:kDisableSetuidSandbox&ss=chromium - "--disable-site-isolation-trials", // https://source.chromium.org/search?q=lang:cpp+symbol:kDisableSiteIsolation&ss=chromium + "--disable-extensions", + "--disable-hang-monitor", + "--disable-ipc-flooding-protection", + "--disable-popup-blocking", + "--disable-prompt-on-repost", + "--disable-renderer-backgrounding", + "--disable-sync", + "--enable-automation", + // TODO(sadym): remove '--enable-blink-features=IdleDetection' once + // IdleDetection is turned on by default. + "--enable-blink-features=IdleDetection", + "--export-tagged-pdf", + "--force-color-profile=srgb", + "--metrics-recording-only", + "--no-first-run", + "--password-store=basic", + "--use-mock-keychain", + ]; + const puppeteerDisableFeatures = [ + "Translate", + "BackForwardCache", + // AcceptCHFrame disabled because of crbug.com/1348106. + "AcceptCHFrame", + "MediaRouter", + "OptimizationHints", + ]; + const puppeteerEnableFeatures = ["NetworkServiceInProcess2"]; + + const chromiumFlags = [ + "--disable-domain-reliability", // https://github.com/GoogleChrome/chrome-launcher/blob/main/docs/chrome-flags-for-tools.md#background-networking + "--disable-print-preview", // https://source.chromium.org/search?q=lang:cpp+symbol:kDisablePrintPreview&ss=chromium "--disable-speech-api", // https://source.chromium.org/search?q=lang:cpp+symbol:kDisableSpeechAPI&ss=chromium - "--disable-web-security", // https://source.chromium.org/search?q=lang:cpp+symbol:kDisableWebSecurity&ss=chromium "--disk-cache-size=33554432", // https://source.chromium.org/search?q=lang:cpp+symbol:kDiskCacheSize&ss=chromium - "--enable-features=SharedArrayBuffer", // https://source.chromium.org/search?q=file:content_features.cc&ss=chromium + "--mute-audio", // https://source.chromium.org/search?q=lang:cpp+symbol:kMuteAudio&ss=chromium + "--no-default-browser-check", // https://source.chromium.org/search?q=lang:cpp+symbol:kNoDefaultBrowserCheck&ss=chromium + "--no-pings", // https://source.chromium.org/search?q=lang:cpp+symbol:kNoPings&ss=chromium + ]; + const chromiumDisableFeatures = [ + "AudioServiceOutOfProcess", + "IsolateOrigins", + "site-per-process", + ]; + const chromiumEnableFeatures = ["SharedArrayBuffer"]; + + const graphicsFlags = [ "--hide-scrollbars", // https://source.chromium.org/search?q=lang:cpp+symbol:kHideScrollbars&ss=chromium "--ignore-gpu-blocklist", // https://source.chromium.org/search?q=lang:cpp+symbol:kIgnoreGpuBlocklist&ss=chromium "--in-process-gpu", // https://source.chromium.org/search?q=lang:cpp+symbol:kInProcessGPU&ss=chromium - "--mute-audio", // https://source.chromium.org/search?q=lang:cpp+symbol:kMuteAudio&ss=chromium - "--no-default-browser-check", // https://source.chromium.org/search?q=lang:cpp+symbol:kNoDefaultBrowserCheck&ss=chromium - "--no-first-run", - "--no-pings", // https://source.chromium.org/search?q=lang:cpp+symbol:kNoPings&ss=chromium - "--no-sandbox", // https://source.chromium.org/search?q=lang:cpp+symbol:kNoSandbox&ss=chromium - "--no-zygote", // https://source.chromium.org/search?q=lang:cpp+symbol:kNoZygote&ss=chromium - "--use-gl=angle", // https://chromium.googlesource.com/chromium/src/+/main/docs/gpu/swiftshader.md - "--use-angle=swiftshader", // https://chromium.googlesource.com/chromium/src/+/main/docs/gpu/swiftshader.md "--window-size=1920,1080", // https://source.chromium.org/search?q=lang:cpp+symbol:kWindowSize&ss=chromium ]; - if (Chromium.headless === true) { - result.push("--single-process"); // https://source.chromium.org/search?q=lang:cpp+symbol:kSingleProcess&ss=chromium - } else { - result.push("--start-maximized"); // https://source.chromium.org/search?q=lang:cpp+symbol:kStartMaximized&ss=chromium - } + // https://chromium.googlesource.com/chromium/src/+/main/docs/gpu/swiftshader.md + this.graphics + ? graphicsFlags.push("--use-gl=angle", "--use-angle=swiftshader") + : graphicsFlags.push("--disable-webgl"); - return result; + const insecureFlags = [ + "--allow-running-insecure-content", // https://source.chromium.org/search?q=lang:cpp+symbol:kAllowRunningInsecureContent&ss=chromium + "--disable-setuid-sandbox", // https://source.chromium.org/search?q=lang:cpp+symbol:kDisableSetuidSandbox&ss=chromium + "--disable-site-isolation-trials", // https://source.chromium.org/search?q=lang:cpp+symbol:kDisableSiteIsolation&ss=chromium + "--disable-web-security", // https://source.chromium.org/search?q=lang:cpp+symbol:kDisableWebSecurity&ss=chromium + "--no-sandbox", // https://source.chromium.org/search?q=lang:cpp+symbol:kNoSandbox&ss=chromium + "--no-zygote", // https://source.chromium.org/search?q=lang:cpp+symbol:kNoZygote&ss=chromium + ]; + + const headlessFlags = [ + this.headless === "new" ? "--headless='new'" : "--headless", + ]; + + return [ + ...puppeteerFlags, + ...chromiumFlags, + `--disable-features=${[ + ...puppeteerDisableFeatures, + ...chromiumDisableFeatures, + ].join(",")}`, + `--enable-features=${[ + ...puppeteerEnableFeatures, + ...chromiumEnableFeatures, + ].join(",")}`, + ...graphicsFlags, + ...insecureFlags, + ...headlessFlags, + ]; } /** - * Returns sensible default viewport settings. + * Returns sensible default viewport settings for serverless environments. */ static get defaultViewport(): Required { return { @@ -208,9 +270,15 @@ class Chromium { return Promise.resolve("/tmp/chromium"); } + /** + * If input is a valid URL, download and extract the file. It will extract to /tmp/chromium-pack + * and executablePath will be recursively called on that location, which will then extract + * the brotli files to the correct locations + */ if (input && isValidUrl(input)) { return this.executablePath(await downloadAndExtract(input)); } + /** * If input is defined, use that as the location of the brotli files, * otherwise, the default location is ../bin. @@ -225,47 +293,75 @@ class Chromium { throw new Error(`The input directory "${input}" does not exist.`); } - const promises = [ - LambdaFS.inflate(`${input}/chromium.br`), - LambdaFS.inflate(`${input}/swiftshader.tar.br`), - ]; - - if ( - process.env.AWS_EXECUTION_ENV !== undefined && - /^AWS_Lambda_nodejs(?:14|16|18)[.]x$/.test( - process.env.AWS_EXECUTION_ENV - ) === true - ) { + // Extract the required files + const promises = [LambdaFS.inflate(`${input}/chromium.br`)]; + if (this.graphics) { + // Only inflate graphics stack if needed + promises.push(LambdaFS.inflate(`${input}/swiftshader.tar.br`)); + } + if (isRunningInAwsLambda()) { + // If running in AWS Lambda, extract more required files promises.push(LambdaFS.inflate(`${input}/aws.tar.br`)); } + // Await all extractions const result = await Promise.all(promises); + // Returns the first result of the promise, which is the location of the `chromium` binary return result.shift() as string; } /** - * Returns a boolean indicating if we are running on AWS Lambda or Google Cloud Functions. - * True is returned if the NODE_ENV is set to 'test' for easier integration testing. - * False is returned if Serverless environment variables `IS_LOCAL` or `IS_OFFLINE` are set. + * Returns the headless mode. + * `true` means the 'old' (legacy, chromium < 112) headless mode. + * "new" means the 'new' headless mode. + * https://developer.chrome.com/articles/new-headless/#try-out-the-new-headless + * @returns true | "new" */ - static get headless() { - if ( - process.env.IS_LOCAL !== undefined || - process.env.IS_OFFLINE !== undefined - ) { - return false; - } - if (process.env.NODE_ENV === "test") { - return true; - } - const environments = [ - "AWS_LAMBDA_FUNCTION_NAME", - "FUNCTION_NAME", - "FUNCTION_TARGET", - "FUNCTIONS_EMULATOR", - ]; + public static get headless() { + return this.headlessMode; + } - return environments.some((key) => process.env[key] !== undefined); + /** + * Sets the headless mode. + * `true` means the 'old' (legacy, chromium < 112) headless mode. + * "new" means the 'new' headless mode. + * https://developer.chrome.com/articles/new-headless/#try-out-the-new-headless + * @default "new" + */ + public static set setHeadlessMode(value: true | "new") { + if ( + (typeof value === "string" && value !== "new") || + (typeof value === "boolean" && value !== true) + ) { + throw new Error( + `Headless mode must be either \`true\` or 'new', you entered '${value}'` + ); + } + this.headlessMode = value; + } + + /** + * Returns whether the graphics stack is enabled or disabled + * @returns boolean + */ + public static get graphics() { + return this.graphicsMode; + } + + /** + * Sets whether the graphics stack is enabled or disabled. + * @param true means the stack is enabled. WebGL will work. + * @param false means that the stack is disabled. WebGL will not work. + * `false` will also skip the extract of the graphics driver, saving about a second during initial extract + * @default true + */ + public static set setGraphicsMode(value: boolean) { + if (typeof value !== "boolean") { + throw new Error( + `Graphics mode must be a boolean, you entered '${value}'` + ); + } + this.graphicsMode = value; } }