Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
499 changes: 494 additions & 5 deletions modules/bitgo/test/v2/unit/internal/tssUtils/ecdsa.ts

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions modules/express/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ BITGO_AUTH_VERSION=2
BITGO_EXTERNAL_SIGNER_URL=
BITGO_SIGNER_MODE=
BITGO_SIGNER_FILE_SYSTEM_PATH=
BITGO_SIGNER_REGISTRY_FILE_PATH=
BITGO_LIGHTNING_SIGNER_FILE_SYSTEM_PATH=
BITGO_KEEP_ALIVE_TIMEOUT=
BITGO_HEADERS_TIMEOUT=
Expand Down
3 changes: 3 additions & 0 deletions modules/express/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,8 @@ This may be preferable for users who would like to apply their signature to thei

For more information please see our [External Signing Mode Documentation](EXTERNAL_SIGNER.md).

When running express in `--signerMode`, one signer process must own each `signerFileSystemPath`: the consumed-share registry file (`<signerFileSystemPath>.consumedShares.json`) is read once per process and rewritten by the process that owns it, so sharing one registry file across replicas can lose replay-protection records. A relay retry after a lost signer response restarts the signing session from the K round (fail-closed). The registry location defaults to `<signerFileSystemPath>.consumedShares.json`; the signer never assumes write access there — if it is not writable, a warning is logged and replay protection degrades to in-memory only (lost on restart). Point `--signerRegistryFilePath` (or `BITGO_SIGNER_REGISTRY_FILE_PATH`) at a writable location for full restart protection.

### KeepAliveTimeout/HeadersTimeout

If the deployment destination is configured with a load balancer such as ELB, a 504 Gateway Timeout may occur. In such cases, it might be resolved by configuring the settings so that LB keepAliveTimeout < keepAliveTimeout < headersTimeout. This can be achieved by specifying the keepAliveTimeout and headersTimeout options.
Expand Down Expand Up @@ -258,6 +260,7 @@ BitGo Express is able to take configuration options from either command line arg
| N/A | --externalSignerUrl | `BITGO_EXTERNAL_SIGNER_URL` | N/A | URL specifying the external API to call for remote signing. |
| N/A | --signerMode | `BITGO_SIGNER_MODE ` | N/A | If set, run Express as a remote signer. |
| N/A | --signerFileSystemPath | `BITGO_SIGNER_FILE_SYSTEM_PATH ` | N/A | Local path specifying where an Express signer machine keeps the encrypted user private keys. Required when signerMode is set. |
| N/A | --signerRegistryFilePath | `BITGO_SIGNER_REGISTRY_FILE_PATH` | N/A | Local path specifying where an Express signer machine keeps the encrypted user private keys. Required when signerMode is set. |
| N/A | --lightningSignerFileSystemPath | `BITGO_LIGHTNING_SIGNER_FILE_SYSTEM_PATH` | N/A | Local path specifying how to contact the lightning signer node. |
| N/A | --keepalivetimeout | `BITGO_KEEP_ALIVE_TIMEOUT` | N/A | BitGo Express server KeepAliveTimeout node. |
| N/A | --headerstimeout | `BITGO_HEADERS_TIMEOUT` | N/A | BitGo Express server HeadersTimeout node. |
Expand Down
4 changes: 4 additions & 0 deletions modules/express/src/args.ts
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,10 @@ parser.addArgument(['--signerFileSystemPath'], {
help: 'Local path specifying where an Express signer machine keeps encrypted user private keys.',
});

parser.addArgument(['--signerRegistryFilePath'], {
help: 'Optional path for the consumed-share registry file (defaults to <signerFileSystemPath>.consumedShares.json). Useful when the key-file directory is read-only.',
});

parser.addArgument(['--lightningSignerFileSystemPath'], {
help: 'Local path specifying where an Express machine keeps lightning signer urls.',
});
Expand Down
43 changes: 41 additions & 2 deletions modules/express/src/clientRoutes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ import { Config } from './config';
import { ApiResponseError, BitGoExpressError } from './errors';
import { promises as fs } from 'fs';
import { retryPromise } from './retryPromise';
import { computeConsumedShareKey, getConsumedShareRegistry } from './consumedSharesRegistry';
import { SENSITIVE_REQUEST_KEYS } from './sensitiveRequestKeys';
import {
handleCreateSignerMacaroon,
Expand Down Expand Up @@ -439,6 +440,31 @@ async function getEncryptedPrivKey(path: string, walletId: string): Promise<stri
return encryptedPrivKey[walletId];
}

/**
* Marks an encrypted share as consumed for this wallet. Each wShare (MuDelta
* round) and oShare (S round) ciphertext may be used exactly once; a replayed
* ciphertext throws before the share is used (fail-closed).
*/
function markConsumedShareClosure(registryFilePath: string, shareType: string, encryptedShare: unknown): () => void {
if (typeof encryptedShare !== 'string' || !encryptedShare) {
throw new ApiResponseError('Missing encrypted share in request', 400);
}
// Cheap pre-checks before the helper runs: an unparseable body or one without string
// iv/ct would fail decryption anyway, and rejecting it early with a 400 keeps junk
// requests from ever reaching the registry.
let envelope: Record<string, unknown>;
try {
envelope = JSON.parse(encryptedShare) as Record<string, unknown>;
} catch {
throw new ApiResponseError('Invalid encrypted share envelope', 400);
}
if (typeof envelope.iv !== 'string' || typeof envelope.ct !== 'string') {
throw new ApiResponseError('Invalid encrypted share envelope', 400);
}
return () =>
getConsumedShareRegistry(registryFilePath).markConsumed(computeConsumedShareKey(shareType, encryptedShare));
}

async function decryptPrivKey(bg: BitGo, encryptedPrivKey: string, walletPw: string): Promise<string> {
try {
return await bg.decrypt({ password: walletPw, input: encryptedPrivKey });
Expand Down Expand Up @@ -467,6 +493,7 @@ export async function handleV2GenerateShareTSS(
if (!signerFileSystemPath) {
throw new Error('Missing required configuration: signerFileSystemPath');
}
const registryFilePath = req.config.signerRegistryFilePath ?? `${signerFileSystemPath}.consumedShares.json`;

const encryptedPrivKey = await getEncryptedPrivKey(signerFileSystemPath, walletId);
const bitgo = req.bitgo;
Expand Down Expand Up @@ -540,10 +567,22 @@ export async function handleV2GenerateShareTSS(
return ecdsaUtils.getOfflineSignerPaillierModulus(req.body);
case ShareType.K:
return await ecdsaUtils.createOfflineKShare(req.body);
case ShareType.MuDelta:
case ShareType.MuDelta: {
req.body.markConsumed = markConsumedShareClosure(
registryFilePath,
req.decoded.sharetype,
req.body.encryptedWShare
);
return await ecdsaUtils.createOfflineMuDeltaShare(req.body);
case ShareType.S:
}
case ShareType.S: {
req.body.markConsumed = markConsumedShareClosure(
registryFilePath,
req.decoded.sharetype,
req.body.encryptedOShare
);
return await ecdsaUtils.createOfflineSShare(req.body);
}
default:
throw new Error(
`Share type ${req.decoded.sharetype} not supported, only PaillierModulus, K, MUDelta, and S share generation is supported.`
Expand Down
5 changes: 5 additions & 0 deletions modules/express/src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ export interface Config {
externalSignerUrl?: string;
signerMode?: boolean;
signerFileSystemPath?: string;
/** Overrides the consumed-share registry location (defaults to `<signerFileSystemPath>.consumedShares.json`). */
signerRegistryFilePath?: string;
lightningSignerFileSystemPath?: string;
keepAliveTimeout?: number;
headersTimeout?: number;
Expand Down Expand Up @@ -76,6 +78,7 @@ export const ArgConfig = (args): Partial<Config> => ({
externalSignerUrl: args.externalSignerUrl,
signerMode: args.signerMode,
signerFileSystemPath: args.signerFileSystemPath,
signerRegistryFilePath: args.signerRegistryFilePath,
lightningSignerFileSystemPath: args.lightningSignerFileSystemPath,
keepAliveTimeout: args.keepalivetimeout,
headersTimeout: args.headerstimeout,
Expand Down Expand Up @@ -103,6 +106,7 @@ export const EnvConfig = (): Partial<Config> => ({
externalSignerUrl: readEnvVar('BITGO_EXTERNAL_SIGNER_URL'),
signerMode: readEnvVar('BITGO_SIGNER_MODE') ? true : undefined,
signerFileSystemPath: readEnvVar('BITGO_SIGNER_FILE_SYSTEM_PATH'),
signerRegistryFilePath: readEnvVar('BITGO_SIGNER_REGISTRY_FILE_PATH'),
lightningSignerFileSystemPath: readEnvVar('BITGO_LIGHTNING_SIGNER_FILE_SYSTEM_PATH'),
keepAliveTimeout: Number(readEnvVar('BITGO_KEEP_ALIVE_TIMEOUT')),
headersTimeout: Number(readEnvVar('BITGO_HEADERS_TIMEOUT')),
Expand Down Expand Up @@ -191,6 +195,7 @@ function mergeConfigs(...configs: Partial<Config>[]): Config {
externalSignerUrl,
signerMode: get('signerMode'),
signerFileSystemPath: get('signerFileSystemPath'),
signerRegistryFilePath: get('signerRegistryFilePath'),
lightningSignerFileSystemPath: get('lightningSignerFileSystemPath'),
keepAliveTimeout: get('keepAliveTimeout'),
headersTimeout: get('headersTimeout'),
Expand Down
171 changes: 171 additions & 0 deletions modules/express/src/consumedSharesRegistry.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
import { createHash } from 'crypto';
import * as fs from 'fs';

import { logger } from '@bitgo/logger';
import { BitGoJsError } from '@bitgo/sdk-core';

import assert from 'assert';

const CONSUMED_SHARES_MAX_AGE_MS = 30 * 24 * 3600 * 1000;

/**
* Mirrors @bitgo/sdk-api decryptV1.ts `ccmNonceLength`: v1 (SJCL/CCM) decryption truncates
* the iv to the nonce length derived from the plaintext size, so trailing iv bytes do not
* affect decryption. Keep in sync with decryptV1.ts.
*/
function v1CcmNonceLength(plaintextLen: number): number {
let L = 2;
while (L < 4 && plaintextLen >= 2 ** (8 * L)) L++;
return 15 - L;
}

/**
* Key excludes attacker-controlled fields (e.g. walletId) so a relabeled replay is still
* rejected. The key covers the canonical decoded bytes that decryption actually consumes:
* for v1 (CCM) the iv is truncated to the effective nonce length (trailing iv bytes are
* ignored by decryption, so any iv longer than the nonce fails closed), and for v2 (GCM) the
* full iv is authenticated. Re-serializing the envelope (key order, whitespace, base64
* padding) still collides with the original key and is rejected.
*/
export function computeConsumedShareKey(shareType: string, encryptedShare: string): string {
let envelope: Record<string, unknown>;
try {
envelope = JSON.parse(encryptedShare) as Record<string, unknown>;
} catch {
return createHash('sha256').update(`${shareType}:${encryptedShare}`).digest('hex');
}
assert(
typeof envelope.iv === 'string' && typeof envelope.ct === 'string',
'encrypted share envelope is missing iv/ct'
);
const iv = Buffer.from(envelope.iv, 'base64');
const ct = Buffer.from(envelope.ct, 'base64');
// v2 (GCM) authenticates the full iv. v1 (CCM) truncates the iv to the nonce length
// (real SJCL envelopes carry a 16-byte iv for a 13-byte nonce), so hash only the
// effective nonce bytes — trailing iv bytes can neither change the key nor affect
// decryption.
let effectiveIv = iv;
if (envelope.v !== 2) {
const tagBytes = typeof envelope.ts === 'number' ? envelope.ts / 8 : 0;
const nonceLen = v1CcmNonceLength(ct.length - tagBytes);
if (iv.length < nonceLen) {
// decryption would reject this envelope anyway; hash the raw string for uniqueness
return createHash('sha256').update(`${shareType}:${encryptedShare}`).digest('hex');
}
effectiveIv = iv.subarray(0, nonceLen);
}
const hash = createHash('sha256');
hash.update(`${shareType}:`);
hash.update(effectiveIv);
hash.update(ct);
return hash.digest('hex');
}
Comment thread
vibhavgo marked this conversation as resolved.
Resolved

/**
* One-time consumption registry: marks wShare/oShare ciphertexts as used before the share runs;
* a second attempt with the same ciphertext throws (fail-closed against replay).
*/
export class ConsumedShareRegistry {
private readonly consumed = new Map<string, number>();
private loaded = false;

constructor(private readonly filePath: string) {}

private load(): void {
if (this.loaded) {
return;
}
let raw: string;
try {
raw = fs.readFileSync(this.filePath, 'utf8');
} catch (e) {
if (typeof e === 'object' && e !== null && 'code' in e && e.code === 'ENOENT') {
// No registry file yet (first run): nothing to load.
this.loaded = true;
return;
}
// An unreadable registry must not silently forget consumed shares: fail closed and
// retry the load on the next call.
const detail = e instanceof Error ? e.message : String(e);
throw new BitGoJsError(`failed to read consumed share registry from ${this.filePath}: ${detail}`);
}
try {
const parsed = JSON.parse(raw) as Record<string, number>;
for (const [key, consumedAt] of Object.entries(parsed)) {
if (typeof consumedAt === 'number') {
this.consumed.set(key, consumedAt);
}
}
this.loaded = true;
} catch {
// A corrupt registry must not silently forget consumed shares: fail closed and retry
// the load on the next call.
throw new BitGoJsError(`failed to parse consumed share registry from ${this.filePath}`);
}
}

markConsumed(key: string): void {
// Check-then-set is race-free only because the whole path is synchronous; do not add an
// await between the check and the set.
this.load();
if (this.consumed.has(key)) {
throw new BitGoJsError('encrypted share has already been consumed; restart the signing session');
}
this.consumed.set(key, Date.now());
this.persist();
}

/** Drops expired entries so the in-memory map and the persisted file stay bounded. */
private prune(): void {
const now = Date.now();
for (const [key, consumedAt] of this.consumed) {
if (now - consumedAt >= CONSUMED_SHARES_MAX_AGE_MS) {
this.consumed.delete(key);
Comment thread
vibhavgo marked this conversation as resolved.
Resolved
}
}
}

private persist(): void {
let tempPath = '';
// Atomic write: a crash mid-write cannot truncate the live registry (a truncated file
// would fail closed on load and lock out all signing).
try {
this.prune();
tempPath = `${this.filePath}.${process.pid}.${Math.random().toString(36).slice(2)}.tmp`;
fs.writeFileSync(tempPath, JSON.stringify(Object.fromEntries(this.consumed)));
fs.renameSync(tempPath, this.filePath);
} catch (e) {
try {
fs.unlinkSync(tempPath);
} catch {
// nothing to clean up
}
// The signer process never assumes file write access (only the separate
// fetchEncryptedPrivKeys provisioning step writes there). If the registry location
// is not writable, degrade to in-memory-only protection: the share stays consumed
// for this process, but persistence (and restart protection) is lost until the
// operator relocates the registry via --signerRegistryFilePath.
const detail = e instanceof Error ? e.message : String(e);
logger.warn(
`failed to persist consumed share registry (${detail}); ` +
`replay protection is in-memory only until ${this.filePath} is writable ` +
'(relocate via --signerRegistryFilePath if needed)'
);
}
}
}

Comment thread
vibhavgo marked this conversation as resolved.
Resolved
const registries = new Map<string, ConsumedShareRegistry>();

export function getConsumedShareRegistry(registryFilePath: string): ConsumedShareRegistry {
let registry = registries.get(registryFilePath);
if (!registry) {
registry = new ConsumedShareRegistry(registryFilePath);
registries.set(registryFilePath, registry);
}
return registry;
}

export function resetConsumedShareRegistriesForTesting(): void {
registries.clear();
}
Loading
Loading