raea/audio

<Audio> plays sound in step with the frame. <Mix> masters everything under it to a loudness and a true-peak ceiling (-14 LUFS and -1 dBTP suit the major platforms). song() and useSong() read the energy map raea song writes, so cuts can land on beats and drops. <Speech> speaks a line, and <Sound3D> places a sound in a 3D scene.

import { staticFile } from "raea";
import { Audio, Mix } from "raea/audio";

function Soundtrack() {
  return (
    <Mix lufs={-14} ceiling={-1}>
      <Audio src={staticFile("media/voice.wav")} bus="voice" />
      <Audio src={staticFile("media/bed.m4a")} bus="music" gainDb={-12} fadeOut={1.5} />
    </Mix>
  );
}

import { Audio, Mix, music, prepareSpeech } from "raea/audio"

raea/audio exports 6 components, 1 hook, 7 functions, 1 value, 25 types. Bevy's counterpart: bevy_audio; raea plays it with seedling on its own mixer. Source: sdk/raea/src/audio.ts.

#Components

#Audio

Sound in step with the frame; no picture. A muted or ended Audio is silent.

Also takes the props of MediaProps.

#Mix

The master: every sound in the composition brought to lufs integrated loudness (-14) under a true-peak limiter at ceiling dBTP (-1.5), as the platforms play back. Put one in the composition, around the soundtrack or anywhere.

#Seedling

A game's sample, played by bevy_seedling on raea's mixer from the frame it first shows until it leaves the page, with its effects in its chain. A render and a live play hear the same. Give a repeated one-shot a new id each time: {hit && <Seedling src={staticFile("hit.wav")} id={hit-${hits}} />}.

#SongPlot

The map drawn for a human to check: section bands named, the energy curve with builds, stops, dips and drops on it, hits, the feature lanes, and the pacing (cuts per bar). raea song FILE --plot OUT.png renders it.

#Sound3D

A sound in the 3D scene: a at a place (at), heard from the scene's camera as it moves, in a render and in a game alike. A moving emitter (a car, a fly, a pan sliding) takes its place frame by frame.

<Sound3D src={staticFile("kettle.wav")} at={onRun(right, 2.7, 0.35, 1.1)} loop volume={0.6} />

Also takes the props of SeedlingProps.

#Speech

A line said on the frames it's rendered: heard from start (made ahead while start is null), stopped at the next word after cut. Keep it rendered until the line ends (speechAt's ended): taken off the page it stops at once (faded over 10 ms).

Also takes the props of Line (text, voice, seed, temperature).

#Hooks

#useSong

Where the song is at the current frame (frame 0 of the enclosing Sequence is offset seconds into the song), with the map bound to the composition's fps as song.

function useSong(map: SongMap, options?: UseSongOptions): SongMoment & { song: Song; }

#Functions

#prepareSpeech

Starts making a line now (or finds it made): call it as soon as the text is known.

function prepareSpeech(line: Line): void

#song

A song map bound to a frame rate (and where frame 0 sits in the song): frames from song times, and the reverse.

function song(map: SongMap, options: SongOptions): Song

#songAt

Where the song is at t seconds: a pure function of the map.

function songAt(map: SongMap, t: number): SongMoment

#speechAt

Where line (heard from start, cut at cut) is at frame: which word, whether it has ended, and where it ends. Pure: a function of the line and the frame, so a game may fold it into its state. Waits for the engine if the line hasn't been made that far yet (a render always waits; live play rarely has to).

function speechAt(line: Line & { start: number; cut?: number | null; }, frame: number, fps: number): SpeechState

#speechDuration

A line's length in seconds (waits until all of it is made; pure).

function speechDuration(line: Line): number

#speechMade

How much of a line has been made so far. Display-only: it depends on the machine and the moment.

function speechMade(line: Line): { seconds: number; done: boolean; }

#speechStart

The frame a line asked for at frame should start: SPEECH_LEAD later.

function speechStart(frame: number, fps: number, lead?: number): number

#Values

#SPEECH_LEAD

How long after a request a line should start, so live play never waits for it (seconds).

#Types

#AudioProps

The props of Audio, tabled there.

#CutOptions

#FrameRange

#Line

#MixProps

The props of Mix, tabled there.

#music

#SeedlingFx

#SeedlingProps

The props of Seedling, tabled there.

#Song

#SongBar

#SongEvent

#SongEventKind

type SongEventKind = "build" | "drop" | "stop" | "hit" | "dip" | "break"

#SongFeatures

#SongHit

A hit inside a stop: time on the beat grid (a beat or half beat), at where the onset is.

#SongLevel

type SongLevel = "intro" | "low" | "mid" | "high" | "outro"

#SongMap

#SongMoment

Where a moment of the song is. Seconds are the song's; infinities when there's no such event.

#SongOptions

#SongPlotProps

The props of SongPlot, tabled there.

#SongSection

#SpeechProps

The props of Speech, tabled there.

#SpeechState

Where a line is at a frame.

#Tone

#UseSongOptions

#VolumeProp

type VolumeProp = number | ((mediaFrame: number) => number)