Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,15 @@ app/
assets/ icons, fonts, images
shared/ app-wide constants, types, pure utils
services/ zip (central directory, Zip64), exe scoring, DLL wrappers,
artwork, recipe, tar + gzip, .run assembly, preflight checks
artwork, recipe, tar + gzip, .run assembly, browser downloads,
preflight checks
stores/ game, artwork, build, preflight: the wizard state
uikit/ presentational components and design tokens
blocks/ ArchivePicker, GameForm, ArtworkGrid, PreflightList, BuildPanel
pages/ Home
```

The UI is written with [nanoviews](https://github.com/TrigenSoftware/nanoviews) and [@nano_kit/store](https://nano-kit.js.org/store). The build runs in Chromium only: it streams the `.run` to disk through `showSaveFilePicker`.
The UI is written with [nanoviews](https://github.com/TrigenSoftware/nanoviews) and [@nano_kit/store](https://nano-kit.js.org/store). Where the browser has `showSaveFilePicker` (Chrome, Edge), the build streams the `.run` to disk through it. Anywhere else (Firefox, Safari, Brave) the browser downloads it as a `Blob` made of the header, meta and the archive `File` itself, so the archive is never read into memory either.

## Development

Expand Down
16 changes: 13 additions & 3 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -508,7 +508,7 @@ The dialog asks **what exactly to remove**, with radio buttons, because the diff

## 6. The web application

A static site with no backend. Chromium only, for streaming to disk (Firefox has no `showSaveFilePicker`).
A static site with no backend. Where the browser has `showSaveFilePicker` (Chrome, Edge), the `.run` is streamed into a file the person picks; anywhere else (Firefox, Safari, and Brave, which switches the picker off) the browser downloads it (6.2).

### 6.1 Reading the zip

Expand Down Expand Up @@ -538,14 +538,24 @@ await file.stream().pipeTo(out, {preventClose: true});

Nothing is held in memory. The progress is by bytes written.

Without the picker the same three parts go into one `Blob`, and the browser downloads it through a link with `download`:

```js
const run = new Blob([header, meta, file], {type: 'application/octet-stream'});
```

The archive goes in as the `File` itself, which a `Blob` keeps as a reference to the disk, so nothing is held in memory here either: the download reads the archive from disk as it goes. The cap of ~1-2 GB applies to a `Blob` built from bytes in memory, not to this one. **Verified** on the built site with a 5 GiB Zip64 archive: in Firefox 157 the download started 0.3 s after the click and took 27 s, the browser grew by 54 MiB with three such downloads running at once, GNU `dd` and `tar` read meta out of the `.run`, and the archive in it matched the original byte for byte. Chromium 156 with the picker removed, as in Brave, did the same in 15 s. Safari is not verified.

The price is control: the person picks no folder unless the browser asks, and the progress is shown by the browser, not by the page. The address of the `Blob` is kept until the next build or another archive; a download already under way goes on without it, verified in both browsers.

### 6.3 Checks before building

All of them warn, and none of them blocks the button:

- there is not a single `.exe` in the archive (the select is empty, there is nothing to build)
- the UTF-8 bit 11 is not set **and** there are non-ASCII names → Cyrillic names will arrive garbled, suggest 7-Zip
- the size is > 4 GB → a reminder about exFAT
- `navigator.storage.estimate()` < the size of the archive → there will not be enough space (Chrome writes into a temporary swap file and swaps it in at the end)
- `navigator.storage.estimate()` < the size of the archive → there will not be enough space (Chrome writes into a temporary swap file and swaps it in at the end). Only where the picker is: Firefox 157 reports the quota of the site instead, 7.4 GiB on a disk with 51 GB free
- the chosen exe got a negative weight under 7.2.1 (the person picked by hand what we ranked down): warn, but do not stand in the way
- the artwork is not filled in

Expand Down Expand Up @@ -797,7 +807,7 @@ Drop a finished `.run` back in → read `recipe.json` from the first 72 KB → r

## 8. Limitations

- **Chromium only.** In Firefox the build would go through a Blob in one piece, capped at ~1-2 GB
- **Without the picker the browser does the saving.** Firefox, Safari and Brave get the `.run` as an ordinary download (6.2): no choice of folder unless the browser asks, no progress on the page, and no warning about free space (6.3)
- **Twice the disk space** while creating the archive on the source machine
- **The file arrives without the execute bit**, and not only from a Windows file system: the browser drops it on any download. Verified on a file downloaded from our own server
- **"Run in Konsole" does not help**, contrary to an earlier edition of this section. Worked out from `/usr/share/kio/servicemenus/konsolerun.desktop`:
Expand Down
42 changes: 39 additions & 3 deletions packages/frontend/app/blocks/BuildPanel/BuildPanel.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,15 @@ import {
import {
InjectionContext,
inject,
provide,
waitTasks
} from 'nanoviews/store'
import { context$ } from 'nanoviews'
import { formatSize } from '~/shared/utils'
import { Build$ } from '~/stores/build'
import {
Build$,
HasSavePicker$
} from '~/stores/build'
import { Game$ } from '~/stores/game'
import { sampleArchive } from '../shared/game.mock.ts'
import buttonStyles from '~/uikit/Button/Button.module.css'
Expand All @@ -33,8 +37,8 @@ const STEPS = [
]

// The panel over a game of its own, with the archive read the way a drop reads it
async function setup(archive: File | null = sampleArchive()) {
const context = new InjectionContext()
async function setup(archive: File | null = sampleArchive(), hasSavePicker = true) {
const context = new InjectionContext([provide(HasSavePicker$, hasSavePicker)])
const game = inject(Game$, context)
const build = inject(Build$, context)

Expand Down Expand Up @@ -63,6 +67,13 @@ function pickFile(name = 'need-for-speed.run') {
return showSaveFilePicker
}

// A browser that downloads the file on its own: the download goes nowhere
function catchDownloads() {
vi.spyOn(URL, 'createObjectURL').mockReturnValue('blob:download')
vi.spyOn(URL, 'revokeObjectURL').mockImplementation(() => {})
vi.spyOn(HTMLAnchorElement.prototype, 'click').mockImplementation(() => {})
}

function buildButton() {
return screen.getByRole<HTMLButtonElement>('button')
}
Expand All @@ -71,6 +82,7 @@ describe('blocks', () => {
describe('BuildPanel', () => {
afterEach(() => {
vi.unstubAllGlobals()
vi.restoreAllMocks()
})

it('should ask for an archive before anything else', async () => {
Expand Down Expand Up @@ -251,6 +263,30 @@ describe('blocks', () => {
expect(screen.queryByRole('list')).toBeNull()
})

describe('without a save picker', () => {
it('should show no progress while building: the browser shows it', async () => {
const { build } = await setup(sampleArchive(), false)

build.$phase('building')

expect(screen.queryByRole('progressbar')).toBeNull()
})

it('should say the browser is downloading the installer once it is handed over', async () => {
const { build } = await setup(sampleArchive(), false)

catchDownloads()
fireEvent.click(buildButton())
await waitTasks(build.$phase)

expect(screen.getByText(
'The browser is downloading need-for-speed.run. Once it is done, copy the file to SteamOS, '
+ 'then, in Desktop Mode:'
)).toBeDefined()
expect(within(screen.getByRole('list')).getAllByRole('listitem').map(step => step.textContent)).toEqual(STEPS)
})
})

it('should keep the class passed by the consumer', () => {
render(() => context$(new InjectionContext())(BuildPanel({
class: 'outer'
Expand Down
34 changes: 32 additions & 2 deletions packages/frontend/app/blocks/BuildPanel/BuildPanel.stories.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,14 @@ import type {
} from '@nanoviews/storybook'
import {
batch,
inject
inject,
provide
} from 'nanoviews/store'
import { context$ } from 'nanoviews'
import {
type BuildPhase,
Build$
Build$,
HasSavePicker$
} from '~/stores/build'
import { withGame } from '../shared/game.mock.ts'
import { BuildPanel } from './BuildPanel.ts'
Expand Down Expand Up @@ -47,6 +50,21 @@ function withBuild({
}
}

/**
* Puts the story in a browser without showSaveFilePicker, as Firefox and
* Safari are, which downloads the file on its own. Listed after withBuild, so
* the build it sets up is the one of this browser.
*
* @returns The decorator.
*/
function withoutSavePicker(): Decorator {
return story => (
context$(provide(HasSavePicker$, false))(
story()
)
)
}

const meta: Meta<typeof BuildPanel> = {
title: 'Blocks/BuildPanel',
component: BuildPanel,
Expand Down Expand Up @@ -82,3 +100,15 @@ export const Done: Story = {
})
]
}

/** Handed to the browser as a download: it shows the progress, the page only what comes next. */
export const Downloading: Story = {
decorators: [
withBuild({
phase: 'done',
progress: 1,
savedAs: 'need-for-speed.run'
}),
withoutSavePicker()
]
}
27 changes: 20 additions & 7 deletions packages/frontend/app/blocks/BuildPanel/BuildPanel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,10 @@ import {
} from 'nanoviews'
import type { ExeCandidate } from '~/services/exe'
import { formatSize } from '~/shared/utils'
import { Build$ } from '~/stores/build'
import {
Build$,
HasSavePicker$
} from '~/stores/build'
import { Game$ } from '~/stores/game'
import { Button } from '~/uikit/Button'
import { ProgressBar } from '~/uikit/ProgressBar'
Expand All @@ -28,8 +31,10 @@ import styles from './BuildPanel.module.css'
export type BuildPanelProps = Attributes<'div'>

interface InstructionsProps {
/** The name of the file the human picked, empty when there is none to tell. */
/** The name the file is saved under, empty when there is none to tell. */
savedAs: Signalish<string>
/** The browser saves the file as a download of its own, which may still be under way. */
downloading: boolean
}

/**
Expand Down Expand Up @@ -59,7 +64,10 @@ function captionOf(progress: number, archive: File | null) {
}

// What to do with the file once it is on disk
const Instructions = component$(({ savedAs }: InstructionsProps) => (
const Instructions = component$(({
savedAs,
downloading
}: InstructionsProps) => (
div({
class: styles.instructions
})(
Expand All @@ -68,7 +76,9 @@ const Instructions = component$(({ savedAs }: InstructionsProps) => (
})(
when(
savedAs,
text`${savedAs} is ready. Copy it to SteamOS, then, in Desktop Mode:`,
downloading
? text`The browser is downloading ${savedAs}. Once it is done, copy the file to SteamOS, then, in Desktop Mode:`
: text`${savedAs} is ready. Copy it to SteamOS, then, in Desktop Mode:`,
'Copy it to SteamOS, then, in Desktop Mode:'
)
),
Expand All @@ -95,7 +105,8 @@ const Instructions = component$(({ savedAs }: InstructionsProps) => (

/**
* Builds the installer and follows it to disk: how far the writing got while
* it lasts, and what to do with the file once it is there.
* it lasts, and what to do with the file once it is there. A browser without
* the picker downloads the file on its own and shows the progress itself.
*/
export const BuildPanel = component$(({
class: className,
Expand All @@ -113,6 +124,7 @@ export const BuildPanel = component$(({
$canBuild,
build
} = inject(Build$)
const hasSavePicker = inject(HasSavePicker$)
// Once the file is written, building it again stays at hand - for a form
// changed since - but is no longer the thing to do
const $done = is($phase, 'done')
Expand All @@ -138,7 +150,7 @@ export const BuildPanel = component$(({
() => summaryOf($selected(), $archive())
)
),
if_($building)(
hasSavePicker && if_($building)(
() => ProgressBar({
label: 'Writing the installer',
value: $progress
Expand All @@ -148,7 +160,8 @@ export const BuildPanel = component$(({
),
if_($done)(
() => Instructions({
savedAs: $savedAs
savedAs: $savedAs,
downloading: !hasSavePicker
})
)
)
Expand Down
11 changes: 8 additions & 3 deletions packages/frontend/app/blocks/PreflightList/PreflightList.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,16 @@ import { render } from '@nanoviews/testing-library'
import {
InjectionContext,
inject,
provide,
waitTasks
} from 'nanoviews/store'
import { context$ } from 'nanoviews'
import type { ArtItem } from '~/services/artwork'
import { Artwork$ } from '~/stores/artwork'
import { Build$ } from '~/stores/build'
import {
Build$,
HasSavePicker$
} from '~/stores/build'
import { Game$ } from '~/stores/game'
import { Preflight$ } from '~/stores/preflight'
import calloutStyles from '~/uikit/Callout/Callout.module.css'
Expand Down Expand Up @@ -71,9 +75,10 @@ function emptySlots(empty: number) {
}

// The list in a context of its own, so the test holds the very stores it shows.
// It comes on screen once the archive is read, as it does on the page.
// It comes on screen once the archive is read, as it does on the page, in a
// browser with the picker, where the free space is asked for.
async function setup(archive: File | null = sampleArchive(), props?: PreflightListProps) {
const context = new InjectionContext()
const context = new InjectionContext([provide(HasSavePicker$, true)])
const game = inject(Game$, context)
const { $notes } = inject(Preflight$, context)

Expand Down
7 changes: 4 additions & 3 deletions packages/frontend/app/file-system-access.d.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
/**
* The File System Access API is not in the TypeScript DOM library yet, and the
* build depends on it: streaming to a picked file is what keeps an 8 GB archive
* out of memory (6.2). Chromium only - see the limitations in section 8.
* The File System Access API is not in the TypeScript DOM library yet. Where a
* browser has it, which only Chromium-based ones do, Brave aside, the build
* streams into the file the human picks; anywhere else the browser downloads
* the file (6.2).
*
* A module that declares into the global scope rather than a script: the
* picker is a browser global, and saying so outright leaves no-implicit-globals
Expand Down
19 changes: 3 additions & 16 deletions packages/frontend/app/pages/Home/Home.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import {
NEW_ISSUE_URL,
REPOSITORY_URL
} from '~/shared/constants'
import { CanSaveFiles$ } from '~/stores/build'
import { HasSavePicker$ } from '~/stores/build'
import { Game$ } from '~/stores/game'
import {
type HomeProps,
Expand All @@ -34,8 +34,6 @@ import styles from './Home.module.css'

const LOW_STORAGE = 'The browser reports about 100 B of free space, less than '
+ 'the archive. The build writes through a temporary file, so it may run out part way.'
const UNSUPPORTED = 'This browser is not supported. Plumber needs a Chromium-based browser, such as '
+ 'Chrome or Edge, to save the installer.'

function dropZone() {
return screen.getByLabelText<HTMLInputElement>('Drop the game archive here')
Expand Down Expand Up @@ -132,18 +130,6 @@ describe('pages', () => {
expect(link.rel).toBe('noopener noreferrer')
})

it('should say so up front when the browser cannot save the installer', async () => {
await setup(null, {}, [provide(CanSaveFiles$, false)])

expect(screen.getByText(UNSUPPORTED)).toBeDefined()
})

it('should not warn about a browser that can save the installer', async () => {
await setup(null, {}, [provide(CanSaveFiles$, true)])

expect(screen.queryByText(UNSUPPORTED)).toBeNull()
})

it('should show how it works ahead of the drop zone', async () => {
await setup(null)

Expand Down Expand Up @@ -245,7 +231,8 @@ describe('pages', () => {
}
})

await setup(sampleArchive())
// Asked for only where the picker is
await setup(sampleArchive(), {}, [provide(HasSavePicker$, true)])

// The free space is asked for once the notes come on screen, with the archive
expect(await screen.findByText(LOW_STORAGE)).toBeDefined()
Expand Down
15 changes: 0 additions & 15 deletions packages/frontend/app/pages/Home/Home.stories.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,7 @@ import type {
Meta,
StoryObj
} from '@nanoviews/storybook'
import { provide } from 'nanoviews/store'
import { context$ } from 'nanoviews'
import { withGame } from '~/blocks/shared/game.mock'
import { CanSaveFiles$ } from '~/stores/build'
import { Home } from './Home.ts'

const meta: Meta<typeof Home> = {
Expand All @@ -29,15 +26,3 @@ export const Default: Story = {}
export const Empty: Story = {
decorators: [withGame(null)]
}

/** Opened in a browser without showSaveFilePicker, as Firefox and Safari are: the page says so first. */
export const UnsupportedBrowser: Story = {
decorators: [
story => (
context$(provide(CanSaveFiles$, false))(
story()
)
),
withGame(null)
]
}
Loading
Loading