diff --git a/README.md b/README.md index f40e06e..cfc4046 100644 --- a/README.md +++ b/README.md @@ -43,14 +43,15 @@ app/ assets/ icons, fonts, images shared/ app-wide constants, types, pure utils services/ zip (central directory, Zip64), exe scoring, DLL wrappers, - artwork, recipe, tar + gzip, .run assembly, preflight checks + artwork, recipe, tar + gzip, .run assembly, browser downloads, + preflight checks stores/ game, artwork, build, preflight: the wizard state uikit/ presentational components and design tokens blocks/ ArchivePicker, GameForm, ArtworkGrid, PreflightList, BuildPanel pages/ Home ``` -The UI is written with [nanoviews](https://github.com/TrigenSoftware/nanoviews) and [@nano_kit/store](https://nano-kit.js.org/store). The build runs in Chromium only: it streams the `.run` to disk through `showSaveFilePicker`. +The UI is written with [nanoviews](https://github.com/TrigenSoftware/nanoviews) and [@nano_kit/store](https://nano-kit.js.org/store). Where the browser has `showSaveFilePicker` (Chrome, Edge), the build streams the `.run` to disk through it. Anywhere else (Firefox, Safari, Brave) the browser downloads it as a `Blob` made of the header, meta and the archive `File` itself, so the archive is never read into memory either. ## Development diff --git a/SPEC.md b/SPEC.md index ab48f23..433ed23 100644 --- a/SPEC.md +++ b/SPEC.md @@ -508,7 +508,7 @@ The dialog asks **what exactly to remove**, with radio buttons, because the diff ## 6. The web application -A static site with no backend. Chromium only, for streaming to disk (Firefox has no `showSaveFilePicker`). +A static site with no backend. Where the browser has `showSaveFilePicker` (Chrome, Edge), the `.run` is streamed into a file the person picks; anywhere else (Firefox, Safari, and Brave, which switches the picker off) the browser downloads it (6.2). ### 6.1 Reading the zip @@ -538,6 +538,16 @@ await file.stream().pipeTo(out, {preventClose: true}); Nothing is held in memory. The progress is by bytes written. +Without the picker the same three parts go into one `Blob`, and the browser downloads it through a link with `download`: + +```js +const run = new Blob([header, meta, file], {type: 'application/octet-stream'}); +``` + +The archive goes in as the `File` itself, which a `Blob` keeps as a reference to the disk, so nothing is held in memory here either: the download reads the archive from disk as it goes. The cap of ~1-2 GB applies to a `Blob` built from bytes in memory, not to this one. **Verified** on the built site with a 5 GiB Zip64 archive: in Firefox 157 the download started 0.3 s after the click and took 27 s, the browser grew by 54 MiB with three such downloads running at once, GNU `dd` and `tar` read meta out of the `.run`, and the archive in it matched the original byte for byte. Chromium 156 with the picker removed, as in Brave, did the same in 15 s. Safari is not verified. + +The price is control: the person picks no folder unless the browser asks, and the progress is shown by the browser, not by the page. The address of the `Blob` is kept until the next build or another archive; a download already under way goes on without it, verified in both browsers. + ### 6.3 Checks before building All of them warn, and none of them blocks the button: @@ -545,7 +555,7 @@ All of them warn, and none of them blocks the button: - there is not a single `.exe` in the archive (the select is empty, there is nothing to build) - the UTF-8 bit 11 is not set **and** there are non-ASCII names → Cyrillic names will arrive garbled, suggest 7-Zip - the size is > 4 GB → a reminder about exFAT -- `navigator.storage.estimate()` < the size of the archive → there will not be enough space (Chrome writes into a temporary swap file and swaps it in at the end) +- `navigator.storage.estimate()` < the size of the archive → there will not be enough space (Chrome writes into a temporary swap file and swaps it in at the end). Only where the picker is: Firefox 157 reports the quota of the site instead, 7.4 GiB on a disk with 51 GB free - the chosen exe got a negative weight under 7.2.1 (the person picked by hand what we ranked down): warn, but do not stand in the way - the artwork is not filled in @@ -797,7 +807,7 @@ Drop a finished `.run` back in → read `recipe.json` from the first 72 KB → r ## 8. Limitations -- **Chromium only.** In Firefox the build would go through a Blob in one piece, capped at ~1-2 GB +- **Without the picker the browser does the saving.** Firefox, Safari and Brave get the `.run` as an ordinary download (6.2): no choice of folder unless the browser asks, no progress on the page, and no warning about free space (6.3) - **Twice the disk space** while creating the archive on the source machine - **The file arrives without the execute bit**, and not only from a Windows file system: the browser drops it on any download. Verified on a file downloaded from our own server - **"Run in Konsole" does not help**, contrary to an earlier edition of this section. Worked out from `/usr/share/kio/servicemenus/konsolerun.desktop`: diff --git a/packages/frontend/app/blocks/BuildPanel/BuildPanel.spec.ts b/packages/frontend/app/blocks/BuildPanel/BuildPanel.spec.ts index c2aa3ae..f579ff8 100644 --- a/packages/frontend/app/blocks/BuildPanel/BuildPanel.spec.ts +++ b/packages/frontend/app/blocks/BuildPanel/BuildPanel.spec.ts @@ -14,11 +14,15 @@ import { import { InjectionContext, inject, + provide, waitTasks } from 'nanoviews/store' import { context$ } from 'nanoviews' import { formatSize } from '~/shared/utils' -import { Build$ } from '~/stores/build' +import { + Build$, + HasSavePicker$ +} from '~/stores/build' import { Game$ } from '~/stores/game' import { sampleArchive } from '../shared/game.mock.ts' import buttonStyles from '~/uikit/Button/Button.module.css' @@ -33,8 +37,8 @@ const STEPS = [ ] // The panel over a game of its own, with the archive read the way a drop reads it -async function setup(archive: File | null = sampleArchive()) { - const context = new InjectionContext() +async function setup(archive: File | null = sampleArchive(), hasSavePicker = true) { + const context = new InjectionContext([provide(HasSavePicker$, hasSavePicker)]) const game = inject(Game$, context) const build = inject(Build$, context) @@ -63,6 +67,13 @@ function pickFile(name = 'need-for-speed.run') { return showSaveFilePicker } +// A browser that downloads the file on its own: the download goes nowhere +function catchDownloads() { + vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:download') + vi.spyOn(URL, 'revokeObjectURL').mockImplementation(() => {}) + vi.spyOn(HTMLAnchorElement.prototype, 'click').mockImplementation(() => {}) +} + function buildButton() { return screen.getByRole('button') } @@ -71,6 +82,7 @@ describe('blocks', () => { describe('BuildPanel', () => { afterEach(() => { vi.unstubAllGlobals() + vi.restoreAllMocks() }) it('should ask for an archive before anything else', async () => { @@ -251,6 +263,30 @@ describe('blocks', () => { expect(screen.queryByRole('list')).toBeNull() }) + describe('without a save picker', () => { + it('should show no progress while building: the browser shows it', async () => { + const { build } = await setup(sampleArchive(), false) + + build.$phase('building') + + expect(screen.queryByRole('progressbar')).toBeNull() + }) + + it('should say the browser is downloading the installer once it is handed over', async () => { + const { build } = await setup(sampleArchive(), false) + + catchDownloads() + fireEvent.click(buildButton()) + await waitTasks(build.$phase) + + expect(screen.getByText( + 'The browser is downloading need-for-speed.run. Once it is done, copy the file to SteamOS, ' + + 'then, in Desktop Mode:' + )).toBeDefined() + expect(within(screen.getByRole('list')).getAllByRole('listitem').map(step => step.textContent)).toEqual(STEPS) + }) + }) + it('should keep the class passed by the consumer', () => { render(() => context$(new InjectionContext())(BuildPanel({ class: 'outer' diff --git a/packages/frontend/app/blocks/BuildPanel/BuildPanel.stories.ts b/packages/frontend/app/blocks/BuildPanel/BuildPanel.stories.ts index 996ce2d..981de91 100644 --- a/packages/frontend/app/blocks/BuildPanel/BuildPanel.stories.ts +++ b/packages/frontend/app/blocks/BuildPanel/BuildPanel.stories.ts @@ -5,11 +5,14 @@ import type { } from '@nanoviews/storybook' import { batch, - inject + inject, + provide } from 'nanoviews/store' +import { context$ } from 'nanoviews' import { type BuildPhase, - Build$ + Build$, + HasSavePicker$ } from '~/stores/build' import { withGame } from '../shared/game.mock.ts' import { BuildPanel } from './BuildPanel.ts' @@ -47,6 +50,21 @@ function withBuild({ } } +/** + * Puts the story in a browser without showSaveFilePicker, as Firefox and + * Safari are, which downloads the file on its own. Listed after withBuild, so + * the build it sets up is the one of this browser. + * + * @returns The decorator. + */ +function withoutSavePicker(): Decorator { + return story => ( + context$(provide(HasSavePicker$, false))( + story() + ) + ) +} + const meta: Meta = { title: 'Blocks/BuildPanel', component: BuildPanel, @@ -82,3 +100,15 @@ export const Done: Story = { }) ] } + +/** Handed to the browser as a download: it shows the progress, the page only what comes next. */ +export const Downloading: Story = { + decorators: [ + withBuild({ + phase: 'done', + progress: 1, + savedAs: 'need-for-speed.run' + }), + withoutSavePicker() + ] +} diff --git a/packages/frontend/app/blocks/BuildPanel/BuildPanel.ts b/packages/frontend/app/blocks/BuildPanel/BuildPanel.ts index bda9a06..e15b71a 100644 --- a/packages/frontend/app/blocks/BuildPanel/BuildPanel.ts +++ b/packages/frontend/app/blocks/BuildPanel/BuildPanel.ts @@ -18,7 +18,10 @@ import { } from 'nanoviews' import type { ExeCandidate } from '~/services/exe' import { formatSize } from '~/shared/utils' -import { Build$ } from '~/stores/build' +import { + Build$, + HasSavePicker$ +} from '~/stores/build' import { Game$ } from '~/stores/game' import { Button } from '~/uikit/Button' import { ProgressBar } from '~/uikit/ProgressBar' @@ -28,8 +31,10 @@ import styles from './BuildPanel.module.css' export type BuildPanelProps = Attributes<'div'> interface InstructionsProps { - /** The name of the file the human picked, empty when there is none to tell. */ + /** The name the file is saved under, empty when there is none to tell. */ savedAs: Signalish + /** The browser saves the file as a download of its own, which may still be under way. */ + downloading: boolean } /** @@ -59,7 +64,10 @@ function captionOf(progress: number, archive: File | null) { } // What to do with the file once it is on disk -const Instructions = component$(({ savedAs }: InstructionsProps) => ( +const Instructions = component$(({ + savedAs, + downloading +}: InstructionsProps) => ( div({ class: styles.instructions })( @@ -68,7 +76,9 @@ const Instructions = component$(({ savedAs }: InstructionsProps) => ( })( when( savedAs, - text`${savedAs} is ready. Copy it to SteamOS, then, in Desktop Mode:`, + downloading + ? text`The browser is downloading ${savedAs}. Once it is done, copy the file to SteamOS, then, in Desktop Mode:` + : text`${savedAs} is ready. Copy it to SteamOS, then, in Desktop Mode:`, 'Copy it to SteamOS, then, in Desktop Mode:' ) ), @@ -95,7 +105,8 @@ const Instructions = component$(({ savedAs }: InstructionsProps) => ( /** * Builds the installer and follows it to disk: how far the writing got while - * it lasts, and what to do with the file once it is there. + * it lasts, and what to do with the file once it is there. A browser without + * the picker downloads the file on its own and shows the progress itself. */ export const BuildPanel = component$(({ class: className, @@ -113,6 +124,7 @@ export const BuildPanel = component$(({ $canBuild, build } = inject(Build$) + const hasSavePicker = inject(HasSavePicker$) // Once the file is written, building it again stays at hand - for a form // changed since - but is no longer the thing to do const $done = is($phase, 'done') @@ -138,7 +150,7 @@ export const BuildPanel = component$(({ () => summaryOf($selected(), $archive()) ) ), - if_($building)( + hasSavePicker && if_($building)( () => ProgressBar({ label: 'Writing the installer', value: $progress @@ -148,7 +160,8 @@ export const BuildPanel = component$(({ ), if_($done)( () => Instructions({ - savedAs: $savedAs + savedAs: $savedAs, + downloading: !hasSavePicker }) ) ) diff --git a/packages/frontend/app/blocks/PreflightList/PreflightList.spec.ts b/packages/frontend/app/blocks/PreflightList/PreflightList.spec.ts index 272cea2..e196e1d 100644 --- a/packages/frontend/app/blocks/PreflightList/PreflightList.spec.ts +++ b/packages/frontend/app/blocks/PreflightList/PreflightList.spec.ts @@ -8,12 +8,16 @@ import { render } from '@nanoviews/testing-library' import { InjectionContext, inject, + provide, waitTasks } from 'nanoviews/store' import { context$ } from 'nanoviews' import type { ArtItem } from '~/services/artwork' import { Artwork$ } from '~/stores/artwork' -import { Build$ } from '~/stores/build' +import { + Build$, + HasSavePicker$ +} from '~/stores/build' import { Game$ } from '~/stores/game' import { Preflight$ } from '~/stores/preflight' import calloutStyles from '~/uikit/Callout/Callout.module.css' @@ -71,9 +75,10 @@ function emptySlots(empty: number) { } // The list in a context of its own, so the test holds the very stores it shows. -// It comes on screen once the archive is read, as it does on the page. +// It comes on screen once the archive is read, as it does on the page, in a +// browser with the picker, where the free space is asked for. async function setup(archive: File | null = sampleArchive(), props?: PreflightListProps) { - const context = new InjectionContext() + const context = new InjectionContext([provide(HasSavePicker$, true)]) const game = inject(Game$, context) const { $notes } = inject(Preflight$, context) diff --git a/packages/frontend/app/file-system-access.d.ts b/packages/frontend/app/file-system-access.d.ts index 31ba032..a9bade5 100644 --- a/packages/frontend/app/file-system-access.d.ts +++ b/packages/frontend/app/file-system-access.d.ts @@ -1,7 +1,8 @@ /** - * The File System Access API is not in the TypeScript DOM library yet, and the - * build depends on it: streaming to a picked file is what keeps an 8 GB archive - * out of memory (6.2). Chromium only - see the limitations in section 8. + * The File System Access API is not in the TypeScript DOM library yet. Where a + * browser has it, which only Chromium-based ones do, Brave aside, the build + * streams into the file the human picks; anywhere else the browser downloads + * the file (6.2). * * A module that declares into the global scope rather than a script: the * picker is a browser global, and saying so outright leaves no-implicit-globals diff --git a/packages/frontend/app/pages/Home/Home.spec.ts b/packages/frontend/app/pages/Home/Home.spec.ts index 6fa684d..2cec5ea 100644 --- a/packages/frontend/app/pages/Home/Home.spec.ts +++ b/packages/frontend/app/pages/Home/Home.spec.ts @@ -24,7 +24,7 @@ import { NEW_ISSUE_URL, REPOSITORY_URL } from '~/shared/constants' -import { CanSaveFiles$ } from '~/stores/build' +import { HasSavePicker$ } from '~/stores/build' import { Game$ } from '~/stores/game' import { type HomeProps, @@ -34,8 +34,6 @@ import styles from './Home.module.css' const LOW_STORAGE = 'The browser reports about 100 B of free space, less than ' + 'the archive. The build writes through a temporary file, so it may run out part way.' -const UNSUPPORTED = 'This browser is not supported. Plumber needs a Chromium-based browser, such as ' - + 'Chrome or Edge, to save the installer.' function dropZone() { return screen.getByLabelText('Drop the game archive here') @@ -132,18 +130,6 @@ describe('pages', () => { expect(link.rel).toBe('noopener noreferrer') }) - it('should say so up front when the browser cannot save the installer', async () => { - await setup(null, {}, [provide(CanSaveFiles$, false)]) - - expect(screen.getByText(UNSUPPORTED)).toBeDefined() - }) - - it('should not warn about a browser that can save the installer', async () => { - await setup(null, {}, [provide(CanSaveFiles$, true)]) - - expect(screen.queryByText(UNSUPPORTED)).toBeNull() - }) - it('should show how it works ahead of the drop zone', async () => { await setup(null) @@ -245,7 +231,8 @@ describe('pages', () => { } }) - await setup(sampleArchive()) + // Asked for only where the picker is + await setup(sampleArchive(), {}, [provide(HasSavePicker$, true)]) // The free space is asked for once the notes come on screen, with the archive expect(await screen.findByText(LOW_STORAGE)).toBeDefined() diff --git a/packages/frontend/app/pages/Home/Home.stories.ts b/packages/frontend/app/pages/Home/Home.stories.ts index d9f7d0c..1106304 100644 --- a/packages/frontend/app/pages/Home/Home.stories.ts +++ b/packages/frontend/app/pages/Home/Home.stories.ts @@ -2,10 +2,7 @@ import type { Meta, StoryObj } from '@nanoviews/storybook' -import { provide } from 'nanoviews/store' -import { context$ } from 'nanoviews' import { withGame } from '~/blocks/shared/game.mock' -import { CanSaveFiles$ } from '~/stores/build' import { Home } from './Home.ts' const meta: Meta = { @@ -29,15 +26,3 @@ export const Default: Story = {} export const Empty: Story = { decorators: [withGame(null)] } - -/** Opened in a browser without showSaveFilePicker, as Firefox and Safari are: the page says so first. */ -export const UnsupportedBrowser: Story = { - decorators: [ - story => ( - context$(provide(CanSaveFiles$, false))( - story() - ) - ), - withGame(null) - ] -} diff --git a/packages/frontend/app/pages/Home/Home.ts b/packages/frontend/app/pages/Home/Home.ts index aaedb50..af2363a 100644 --- a/packages/frontend/app/pages/Home/Home.ts +++ b/packages/frontend/app/pages/Home/Home.ts @@ -21,7 +21,6 @@ import { NEW_ISSUE_URL, REPOSITORY_URL } from '~/shared/constants' -import { CanSaveFiles$ } from '~/stores/build' import { Game$ } from '~/stores/game' import { Callout } from '~/uikit/Callout' import mixins from '~/uikit/mixins.module.css' @@ -71,7 +70,6 @@ export const Home = component$(({ ...props }: HomeProps) => { const { $archive } = inject(Game$) - const canSave = inject(CanSaveFiles$) return ( main({ @@ -125,14 +123,6 @@ export const Home = component$(({ 'open an issue on GitHub' ), '.' - ), - // Anywhere else the archive can be read but the installer never saved, - // so the page says so before anything starts - !canSave && Callout({ - tone: 'error' - })( - 'This browser is not supported. Plumber needs a Chromium-based browser, such as ' - + 'Chrome or Edge, to save the installer.' ) ), // Ahead of the drop zone: the first steps are done before the page is diff --git a/packages/frontend/app/services/bundle/bundle.spec.ts b/packages/frontend/app/services/bundle/bundle.spec.ts index 5bb7e20..180ad70 100644 --- a/packages/frontend/app/services/bundle/bundle.spec.ts +++ b/packages/frontend/app/services/bundle/bundle.spec.ts @@ -9,6 +9,7 @@ import { buildHeader, buildMeta, packMeta, + runBlob, writeRunFile } from './bundle.ts' @@ -234,5 +235,34 @@ describe('services', () => { expect(sink.chunks.flatMap(chunk => [...chunk])).toHaveLength(BLOCK_SIZE) }) }) + + describe('runBlob', () => { + it('should lay out header, meta and payload in that order', async () => { + const payload = new File([new Uint8Array([9, 9, 9])], 'game.zip') + const meta = await buildMeta(RECIPE, []) + const header = buildHeader(meta.byteLength / BLOCK_SIZE) + const run = [ + ...new Uint8Array(await runBlob({ + header, + meta, + payload + }).arrayBuffer()) + ] + + expect(run.slice(0, BLOCK_SIZE)).toEqual([...header]) + expect(run.slice(BLOCK_SIZE, BLOCK_SIZE + meta.byteLength)).toEqual([...meta]) + expect(run.slice(BLOCK_SIZE + meta.byteLength)).toEqual([9, 9, 9]) + }) + + it('should be plain bytes, for the browser to save rather than open', async () => { + const meta = await buildMeta(RECIPE, []) + + expect(runBlob({ + header: buildHeader(1), + meta, + payload: new Blob() + }).type).toBe('application/octet-stream') + }) + }) }) }) diff --git a/packages/frontend/app/services/bundle/bundle.ts b/packages/frontend/app/services/bundle/bundle.ts index 646bf6a..943c281 100644 --- a/packages/frontend/app/services/bundle/bundle.ts +++ b/packages/frontend/app/services/bundle/bundle.ts @@ -11,6 +11,7 @@ import { } from '~/services/tar' import type { BuildProgress, + RunBlobParts, RunParts } from './bundle.types.ts' @@ -176,3 +177,19 @@ export async function* writeRunFile( writer.releaseLock() } } + +/** + * The `.run` for a browser that can only download it: header, meta and the + * archive in one Blob. The archive goes in as the File itself, which a Blob + * keeps as a reference to the disk, and the download reads it from there as it + * goes, so an 8 GB game costs no more memory than a small one here either + * (6.2). + * + * @returns The Blob, typed as plain bytes, so the browser saves it rather than + * opens it. + */ +export function runBlob(parts: RunBlobParts) { + return new Blob([parts.header, parts.meta, parts.payload], { + type: 'application/octet-stream' + }) +} diff --git a/packages/frontend/app/services/bundle/bundle.types.ts b/packages/frontend/app/services/bundle/bundle.types.ts index 65fe4bb..b56ef03 100644 --- a/packages/frontend/app/services/bundle/bundle.types.ts +++ b/packages/frontend/app/services/bundle/bundle.types.ts @@ -5,6 +5,13 @@ export interface RunParts { payloadSize: number } +export interface RunBlobParts { + header: Uint8Array + meta: Uint8Array + /** The archive as the File itself, not its bytes. */ + payload: Blob +} + export interface BuildProgress { written: number total: number diff --git a/packages/frontend/app/services/download/download.spec.ts b/packages/frontend/app/services/download/download.spec.ts new file mode 100644 index 0000000..32f86a7 --- /dev/null +++ b/packages/frontend/app/services/download/download.spec.ts @@ -0,0 +1,67 @@ +import { + vi, + describe, + it, + expect, + afterEach +} from 'vitest' +import { startDownload } from './download.ts' + +// The browser side of a download: the address it gives the Blob, the click +// that would save it, and letting the address go +function catchDownload() { + return { + createObjectURL: vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:run'), + revokeObjectURL: vi.spyOn(URL, 'revokeObjectURL').mockImplementation(() => {}), + click: vi.spyOn(HTMLAnchorElement.prototype, 'click').mockImplementation(() => {}) + } +} + +describe('services', () => { + describe('download', () => { + describe('startDownload', () => { + afterEach(() => { + vi.restoreAllMocks() + }) + + it('should click a link to the Blob, named for the file to save', () => { + const { + createObjectURL, + click + } = catchDownload() + const blob = new Blob(['#!/bin/sh\n']) + + startDownload(blob, 'need-for-speed.run') + + // The link is what the click was called on + const [link] = click.mock.contexts as HTMLAnchorElement[] + + expect(createObjectURL).toHaveBeenCalledExactlyOnceWith(blob) + expect(click).toHaveBeenCalledOnce() + expect(link.href).toBe('blob:run') + expect(link.download).toBe('need-for-speed.run') + }) + + it('should keep the address of the Blob until it is let go', () => { + const { revokeObjectURL } = catchDownload() + const letGo = startDownload(new Blob(), 'need-for-speed.run') + + expect(revokeObjectURL).not.toHaveBeenCalled() + + letGo() + + expect(revokeObjectURL).toHaveBeenCalledExactlyOnceWith('blob:run') + }) + + it('should let the address go only once', () => { + const { revokeObjectURL } = catchDownload() + const letGo = startDownload(new Blob(), 'need-for-speed.run') + + letGo() + letGo() + + expect(revokeObjectURL).toHaveBeenCalledOnce() + }) + }) + }) +}) diff --git a/packages/frontend/app/services/download/download.ts b/packages/frontend/app/services/download/download.ts new file mode 100644 index 0000000..d220edf --- /dev/null +++ b/packages/frontend/app/services/download/download.ts @@ -0,0 +1,24 @@ +/** + * Hands a Blob to the browser to save under the name given, as a link with + * `download` does. + * + * @returns Lets the address of the Blob go, once: calling it again does + * nothing. When a browser takes it up after the click is its own affair, so + * the caller lets it go no sooner than the download is surely under way; one + * already under way goes on without it. + */ +export function startDownload(blob: Blob, name: string) { + let url = URL.createObjectURL(blob) + const link = document.createElement('a') + + link.href = url + link.download = name + link.click() + + return () => { + if (url) { + URL.revokeObjectURL(url) + url = '' + } + } +} diff --git a/packages/frontend/app/services/download/index.ts b/packages/frontend/app/services/download/index.ts new file mode 100644 index 0000000..4b4055c --- /dev/null +++ b/packages/frontend/app/services/download/index.ts @@ -0,0 +1 @@ +export * from './download.ts' diff --git a/packages/frontend/app/stores/build.spec.ts b/packages/frontend/app/stores/build.spec.ts index e9ff509..5aec817 100644 --- a/packages/frontend/app/stores/build.spec.ts +++ b/packages/frontend/app/stores/build.spec.ts @@ -9,6 +9,7 @@ import { InjectionContext, effect, inject, + provide, start, waitTasks } from '@nano_kit/store' @@ -22,7 +23,7 @@ import { import { Artwork$ } from './artwork.ts' import { Build$, - CanSaveFiles$ + HasSavePicker$ } from './build.ts' import { Game$ } from './game.ts' @@ -77,6 +78,35 @@ function pickFile(name = 'need-for-speed.run') { } } +// A picker that breaks as it opens: something for a build to fail with +function breakPicker() { + vi.stubGlobal('showSaveFilePicker', () => Promise.reject(new Error('The picker broke.'))) +} + +// A browser that saves the file as a download of its own: the Blob it is +// handed and the link it is clicked through stay here to be looked at +function catchDownloads() { + const blobs: Blob[] = [] + const revoke = vi.spyOn(URL, 'revokeObjectURL').mockImplementation(() => {}) + const click = vi.spyOn(HTMLAnchorElement.prototype, 'click').mockImplementation(() => {}) + + vi.spyOn(URL, 'createObjectURL').mockImplementation((blob) => { + blobs.push(blob as Blob) + + return `blob:download-${blobs.length}` + }) + + return { + revoke, + // The links themselves, as what the clicks were called on + clicked: () => (click.mock.contexts as HTMLAnchorElement[]).map(link => ({ + href: link.href, + download: link.download + })), + downloaded: async (index = 0) => new Uint8Array(await blobs[index].arrayBuffer()) + } +} + // meta.tar.gz unpacked the way GNU gzip on SteamOS reads it: the decoder here // takes the padding and the archive after the gzip stream for junk and errors, // but only once it has handed out everything meta holds @@ -122,8 +152,8 @@ function filesOf(tar: Uint8Array) { // The form and the build panel on screen, as they are while a build is set up const stops: (() => void)[] = [] -async function setup(file = archiveOf()) { - const context = new InjectionContext() +async function setup(file = archiveOf(), hasSavePicker = true) { + const context = new InjectionContext([provide(HasSavePicker$, hasSavePicker)]) const game = inject(Game$, context) const artwork = inject(Artwork$, context) const build = inject(Build$, context) @@ -140,7 +170,7 @@ async function setup(file = archiveOf()) { } describe('stores', () => { - describe('CanSaveFiles$', () => { + describe('HasSavePicker$', () => { afterEach(() => { vi.unstubAllGlobals() }) @@ -148,18 +178,19 @@ describe('stores', () => { it('should say yes where the browser has showSaveFilePicker', () => { vi.stubGlobal('showSaveFilePicker', () => Promise.reject(new Error('Not picked in a test'))) - expect(inject(CanSaveFiles$, new InjectionContext())).toBe(true) + expect(inject(HasSavePicker$, new InjectionContext())).toBe(true) }) it('should say no where it has none, as Firefox and Safari do', () => { // happy-dom has none either - expect(inject(CanSaveFiles$, new InjectionContext())).toBe(false) + expect(inject(HasSavePicker$, new InjectionContext())).toBe(false) }) }) describe('Build$', () => { afterEach(() => { vi.unstubAllGlobals() + vi.restoreAllMocks() stops.splice(0).forEach(stop => stop()) }) @@ -266,6 +297,7 @@ describe('stores', () => { it('should forget the last failure once another build starts', async () => { const { build } = await setup() + breakPicker() build.build() await waitTasks(build.$phase) @@ -402,16 +434,6 @@ describe('stores', () => { expect(build.$error()).toBe('There is not enough space on the disk.') }) - it('should fail with the reason in a browser without the file picker', async () => { - const { build } = await setup() - - build.build() - await waitTasks(build.$phase) - - expect(build.$phase()).toBe('failed') - expect(build.$error()).toContain('showSaveFilePicker') - }) - it('should start over when the archive changes', async () => { const { game, build } = await setup() @@ -428,6 +450,7 @@ describe('stores', () => { it('should forget why the last build failed when another archive comes', async () => { const { game, build } = await setup() + breakPicker() build.build() await waitTasks(build.$phase) game.$file(archiveOf('Hot Pursuit 2.zip')) @@ -474,5 +497,138 @@ describe('stores', () => { expect(build.$progress()).toBe(0) expect(build.$savedAs()).toBe('') }) + + describe('without a save picker', () => { + it('should hand the installer to the browser to download under the name of the game', async () => { + const { build } = await setup(archiveOf(), false) + const { clicked } = catchDownloads() + + build.build() + await waitTasks(build.$phase) + + expect(clicked()).toEqual([ + { + href: 'blob:download-1', + download: 'need-for-speed.run' + } + ]) + }) + + it('should not open a file picker, even one the browser has', async () => { + const { build } = await setup(archiveOf(), false) + const { showSaveFilePicker } = pickFile() + + catchDownloads() + build.build() + await waitTasks(build.$phase) + + expect(showSaveFilePicker).not.toHaveBeenCalled() + }) + + it('should download header, meta and the archive', async () => { + const file = archiveOf() + const { build } = await setup(file, false) + const { downloaded } = catchDownloads() + + build.build() + await waitTasks(build.$phase) + + const run = await downloaded() + const archive = new Uint8Array(await file.arrayBuffer()) + + expect(decoder.decode(run.subarray(0, 10))).toBe('#!/bin/sh\n') + expect((run.byteLength - archive.byteLength) % BLOCK_SIZE).toBe(0) + expect(run.subarray(run.byteLength - archive.byteLength)).toEqual(archive) + }) + + it('should pack the recipe the human settled on', async () => { + const { game, build } = await setup(archiveOf(), false) + const { downloaded } = catchDownloads() + + game.$name('Hot Pursuit 2') + build.build() + await waitTasks(build.$phase) + + const recipe = filesOf(await unpackMeta(await downloaded())).get('./recipe.json') + + expect(JSON.parse(decoder.decode(recipe))).toMatchObject({ + name: 'Hot Pursuit 2', + exe: 'Need for Speed/speed.exe' + }) + }) + + it('should be done once the download is handed over, under the name it was given', async () => { + const { build } = await setup(archiveOf(), false) + + catchDownloads() + build.build() + await waitTasks(build.$phase) + + expect(build.$phase()).toBe('done') + expect(build.$progress()).toBe(1) + expect(build.$savedAs()).toBe('need-for-speed.run') + expect(build.$error()).toBeNull() + }) + + it('should keep the last download until another build is handed over', async () => { + const { build } = await setup(archiveOf(), false) + const { revoke } = catchDownloads() + + build.build() + await waitTasks(build.$phase) + + expect(revoke).not.toHaveBeenCalled() + + build.build() + await waitTasks(build.$phase) + + expect(revoke).toHaveBeenCalledExactlyOnceWith('blob:download-1') + }) + + it('should let the download go when the archive changes', async () => { + const { game, build } = await setup(archiveOf(), false) + const { revoke } = catchDownloads() + + build.build() + await waitTasks(build.$phase) + game.$file(null) + + expect(revoke).toHaveBeenCalledExactlyOnceWith('blob:download-1') + }) + + it('should download nothing for a build of an archive no longer there', async () => { + const { game, build } = await setup(archiveOf(), false) + const { clicked } = catchDownloads() + + build.build() + game.$file(null) + await waitTasks(build.$phase) + + expect(clicked()).toEqual([]) + expect(build.$phase()).toBe('idle') + expect(build.$savedAs()).toBe('') + }) + + it('should fail with the reason when the installer cannot be put together', async () => { + const { artwork, build } = await setup(archiveOf(), false) + const unreadable = new File([], 'cover.png', { + type: 'image/png' + }) + + catchDownloads() + unreadable.arrayBuffer = () => Promise.reject(new Error('cover.png could not be read.')) + artwork.$slots({ + portrait: { + ...COVER, + file: unreadable + } + }) + build.build() + await waitTasks(build.$phase) + + expect(build.$phase()).toBe('failed') + expect(build.$error()).toBe('cover.png could not be read.') + }) + }) }) }) diff --git a/packages/frontend/app/stores/build.ts b/packages/frontend/app/stores/build.ts index ac03cbc..3ea302c 100644 --- a/packages/frontend/app/stores/build.ts +++ b/packages/frontend/app/stores/build.ts @@ -8,6 +8,7 @@ import { task, inject } from '@nano_kit/store' +import { startDownload } from '~/services/download' import { buildRecipe, slugify @@ -20,21 +21,22 @@ import { Game$ } from './game.ts' export type BuildPhase = 'idle' | 'building' | 'done' | 'failed' /** - * Whether this browser can save the installer at all: the build streams the - * `.run` to disk through showSaveFilePicker, which only Chromium-based browsers - * have (8). A token rather than a check in place, so a story or a test can - * stand in for either kind of browser. + * Whether this browser has showSaveFilePicker, which only Chromium-based ones + * have, and Brave switches off: with it the build streams the `.run` into the + * file the human picks and follows the bytes as they go, without it the + * browser downloads the file on its own (6.2). A token rather than a check in + * place, so a story or a test can stand in for either kind of browser. * * @returns Whether the picker is there. */ -export function CanSaveFiles$() { +export function HasSavePicker$() { return typeof showSaveFilePicker === 'function' } /** - * Builds the `.run` and streams it straight to the file the human picked. - * Chromium only: Firefox has no showSaveFilePicker, and building through a - * Blob would cap the size at a gigabyte or two (8). + * Builds the `.run` and saves it: straight into the file the human picked + * where the browser has the picker, as a download of the browser's own + * elsewhere. Either way the archive is never held in memory (6.2). * * @returns The store. */ @@ -52,10 +54,12 @@ export function Build$() { $problems, $error: $artworkError } = inject(Artwork$) + const hasSavePicker = inject(HasSavePicker$) const $phase = mountable(signal('idle')) // A share of the whole file, from 0 to 1 const $progress = signal(0) - // The name of the file the human picked, once it is written + // The name the file is saved under, once it is: the one the human picked, + // or the one the download was given const $savedAs = signal('') // Why the last build failed const $error = mountable(signal(null)) @@ -69,6 +73,9 @@ export function Build$() { )) // A build of an archive that is no longer there writes nothing back let ticket = 0 + // Lets the last download go: no sooner than the next build or archive, by + // when it is surely under way + let releaseDownload: (() => void) | null = null const run = async (runTicket: number) => { const isCurrent = () => runTicket === ticket @@ -91,6 +98,7 @@ export function Build$() { userData: $userDataList(), dllOverrides: $overrides() }) + const fileName = `${slugify(recipe.name)}.run` // What went wrong is told about the last attempt: a new build clears an // image that could not be read as well, as it always did @@ -103,44 +111,58 @@ export function Build$() { // Asked before anything is awaited: the user activation of the click does // not survive an await, and without it the picker refuses to open. - const handle = await showSaveFilePicker({ - suggestedName: `${slugify(recipe.name)}.run`, - types: [ - { - description: 'Self-extracting installer', - accept: { - 'application/x-sh': ['.run'] + const handle = hasSavePicker + ? await showSaveFilePicker({ + suggestedName: fileName, + types: [ + { + description: 'Self-extracting installer', + accept: { + 'application/x-sh': ['.run'] + } } - } - ] - }) + ] + }) + : null // Loaded only now: the tar writer and the header are needed by the build // alone, so they stay out of what the page loads up front, like the // installer scripts they pack. Not before the picker, which needs the click. const { buildHeader, buildMeta, + runBlob, writeRunFile } = await import('~/services/bundle') const meta = await buildMeta(recipe, items) const header = buildHeader(meta.byteLength / BLOCK_SIZE) - for await (const { written, total } of writeRunFile(await handle.createWritable(), { - header, - meta, - payload: archive.stream(), - payloadSize: archive.size - })) { - if (isCurrent()) { - $progress(written / total) + if (handle) { + for await (const { written, total } of writeRunFile(await handle.createWritable(), { + header, + meta, + payload: archive.stream(), + payloadSize: archive.size + })) { + if (isCurrent()) { + $progress(written / total) + } } + } else if (isCurrent()) { + // Nothing is on disk before the download, so a build of an archive no + // longer there leaves nothing behind + releaseDownload?.() + releaseDownload = startDownload(runBlob({ + header, + meta, + payload: archive + }), fileName) } if (isCurrent()) { batch(() => { $phase('done') $progress(1) - $savedAs(handle.name) + $savedAs(handle?.name ?? fileName) }) } } catch (error) { @@ -184,6 +206,7 @@ export function Build$() { } ticket += 1 + releaseDownload?.() batch(() => { $phase('idle') $progress(0) diff --git a/packages/frontend/app/stores/preflight.spec.ts b/packages/frontend/app/stores/preflight.spec.ts index d329fd9..22f06ed 100644 --- a/packages/frontend/app/stores/preflight.spec.ts +++ b/packages/frontend/app/stores/preflight.spec.ts @@ -9,6 +9,7 @@ import { STORE_UNMOUNT_DELAY, InjectionContext, inject, + provide, start, waitTasks } from '@nano_kit/store' @@ -18,7 +19,10 @@ import { makeZip } from '~/services/zip/zip.mock' import { Artwork$ } from './artwork.ts' -import { Build$ } from './build.ts' +import { + Build$, + HasSavePicker$ +} from './build.ts' import { Game$ } from './game.ts' import { Preflight$ } from './preflight.ts' @@ -66,8 +70,8 @@ function archiveOf(name = 'Need for Speed.zip', entries = ENTRIES) { // What is on screen: the summary of the archive always, the checks when a test shows them const stops: (() => void)[] = [] -function setup() { - const context = new InjectionContext() +function setup(hasSavePicker = true) { + const context = new InjectionContext([provide(HasSavePicker$, hasSavePicker)]) const game = inject(Game$, context) stops.push(start(game.$archive)) @@ -210,6 +214,18 @@ describe('stores', () => { expect(estimate).not.toHaveBeenCalled() }) + it('should not ask for the free space where the browser downloads the file itself', async () => { + const { game, preflight } = setup(false) + const estimate = reportStorage(1_000, 900) + + stops.push(start(preflight.$checks)) + game.$file(archiveOf()) + await waitTasks(preflight.$checks) + + expect(estimate).not.toHaveBeenCalled() + expect(preflight.$checks()).toEqual([emptySlots(5)]) + }) + it('should ask for the free space again when another archive comes', async () => { const { game, preflight } = setup() const estimate = reportStorage(1_000_000_000, 0) diff --git a/packages/frontend/app/stores/preflight.ts b/packages/frontend/app/stores/preflight.ts index 3ad5cf5..6486f93 100644 --- a/packages/frontend/app/stores/preflight.ts +++ b/packages/frontend/app/stores/preflight.ts @@ -14,7 +14,10 @@ import { preflight } from '~/services/checks' import { Artwork$ } from './artwork.ts' -import { Build$ } from './build.ts' +import { + Build$, + HasSavePicker$ +} from './build.ts' import { Game$ } from './game.ts' export type NoteLevel = 'error' | CheckLevel @@ -50,6 +53,7 @@ export function Preflight$() { $error: $artworkError } = inject(Artwork$) const { $error: $buildError } = inject(Build$) + const hasSavePicker = inject(HasSavePicker$) /** From navigator.storage.estimate(), or null when it could not be asked. */ const $storageAvailable = mountable(signal(null)) const measure = action(async () => { @@ -101,11 +105,16 @@ export function Preflight$() { return notes }) - // Measured while the checks are on screen, afresh for every archive - onMountEffect($storageAvailable, () => { - $archive() - void task($storageAvailable, measure()) - }) + // Measured while the checks are on screen, afresh for every archive. Only + // where the picker is: the check stands on the estimate Chromium gives (6.3), + // and Firefox gives the quota of the site instead, a few GB however much the + // disk the download goes to has free + if (hasSavePicker) { + onMountEffect($storageAvailable, () => { + $archive() + void task($storageAvailable, measure()) + }) + } return { $storageAvailable, diff --git a/packages/frontend/vite.config.ts b/packages/frontend/vite.config.ts index 5d2acaf..b7c9836 100644 --- a/packages/frontend/vite.config.ts +++ b/packages/frontend/vite.config.ts @@ -17,8 +17,9 @@ export default defineConfig({ lightningcss: { // Vite's default targets, the widely available baseline, would compile // light-dark() into variables switched by prefers-color-scheme, which a - // color-scheme forced on the root cannot reach. The build is Chromium-only - // (showSaveFilePicker), and Chromium has light-dark() natively + // color-scheme forced on the root cannot reach. The browsers the page is + // for have light-dark() natively: Chromium since 123, Firefox since 120, + // Safari since 17.5 exclude: Features.LightDark, drafts: { customMedia: true