AudioStream

AudioStream owns a single cubeb stream and turns the backend’s raw frame counter into a media-time clock. It is owned by AudioSink, which is wrapped by AudioSinkWrapper, and that wrapper is what MediaDecoderStateMachine reads the playback position from. So the clock described here is what the video sink synchronises against.

Two responsibilities live here and are worth keeping apart: handing decoded audio to the backend, on the backend’s realtime callback thread, and reporting a media position, on the state machine’s thread. AudioClock and its FrameHistory are the bridge between them.

The two frame cursors

A backend exposes two frame counts, and they do not mean the same thing.

• The write cursor is what the data callback has handed over. It moves in one jump per callback, then sits still.

• The play cursor is what the device has actually output, read through cubeb_stream_get_position. It rises continuously, driven by the hardware clock.

The gap between them is audio handed over but not yet heard. This document calls it the unplayed audio.

frames ---------------------------------------------------------------->

                     P                                W
                     |                                |
  ..... played ......|........... unplayed ...........|
                     |<------------------------------>|
                     |  handed to the backend,        |
                     |  not yet heard                 |

  P = play cursor    (cubeb_stream_get_position)
  W = write cursor   (what the data callback has handed over)
  unplayed = W - P

The unplayed audio is the cushion that lets a slightly late callback still find audio queued ahead of it, so it does not reach zero during healthy playback. Its size is the device’s output latency plus the part of the last callback the backend has not yet interpolated past, so it jumps up by one callback on each callback and slides back down to the latency in between.

Two consequences matter below:

• The play cursor normally stays at or below the write cursor, because it reports what the device has consumed and a device cannot consume what nobody handed it. How it is derived varies: some backends compute it as the frames handed over minus the latency they report, others return an OS or device counter. Either way it is an estimate rather than a hardware readout, and some backends extrapolate from the wall clock since the last callback, so a late callback or a device xrun can report it above the write cursor. The frame history can also account for less than the device has played, because a callback that found the queue full keeps its frames on the audio thread until a later call collects them.

• The unplayed audio belongs to the position that was current when it was handed over, not to anything that happens afterwards.

The frame history

FrameHistory maps a play-cursor value to a media time. It is a piecewise mapping rather than a single division, because silence carries no media time and the playback rate can change mid-stream.

Member	Unit	Meaning
mBaseOffset	frames	Where the retained history starts
mBasePosition	microseconds	The media time at mBaseOffset
mChunks	both	Segments from mBaseOffset upward

The first two are one anchor in two units: when the device had output mBaseOffset frames, the media was at mBasePosition. GetPosition starts there and walks the chunks up to the play cursor.

Per chunk, servicedFrames counts frames carrying media time and totalFrames counts every frame output, so the difference is silence inserted on an underrun. The clamp min(delta, servicedFrames) holds the clock still while that silence is audible, and each chunk’s rate is the output rate in force when it was recorded, so a rate change maps correctly on both sides of it.

The conserved identity

mBaseOffset starts at zero and folding a chunk away adds its totalFrames to it, so this sum is invariant:

mBaseOffset + sum of every retained chunk's totalFrames  ==  the write cursor

A query normally lands inside the retained chunks, because the play cursor stays at or below the write cursor. The exception is a callback still stranded on the audio thread, which leaves the history short. Keeping the identity true is the whole job: break it and the walk runs off the end, at which point GetPosition falls back to mBasePosition alone, which is the serviced total, so the reported position stops following the device between callbacks and jumps instead.

Rebasing a stream reused across a seek

When the stream is kept alive across a seek rather than destroyed, its frame counter keeps climbing while media time has to restart, so RebaseLive moves the anchor instead of resetting the counter.

Media time zero belongs at the write cursor, not the play cursor, because everything below the write cursor was handed over before the seek. The anchor cannot simply be set to the write cursor: the history stores playCursor - mBaseOffset in a uint32_t and asserts the difference is not negative, so an anchor above the play cursor trips that assertion in a debug build and wraps in an opt one. The anchor therefore stays at the play cursor and the unplayed audio is carried across as a chunk that services nothing:

frames:       P                                W
              |                                |
              |<---------- unplayed ---------->|
              |     one chunk, totalFrames =   |
              |     unplayed, servicedFrames   |
              |     = 0                        |
media time:   0 .............................. 0 ------> counts up from here

totalFrames keeps the identity intact so queries stay inside the chunks, and servicedFrames of zero means draining that audio advances no media time, so the clock holds at the seek target until the first post-seek frame is heard. Both come from one subtraction, W - P.

W is a counter the clock keeps itself, bumped on the audio thread, not a sum over the retained chunks: the history is only fed when the owner thread drains the callback queue, and nothing drains it during a seek. It is read after synchronising with that thread, since an early read understates the gap and lets the position lead the audio for the rest of playback.

The subtraction needs both cursors, and the audio it describes has to be audio that will still be played:

Situation	Carried
Stream kept running across the seek	W - P
Backend restarted, queue discarded	nothing
Position query failed, no P	nothing
P reported above W	nothing
Backend drained since the resume	nothing

Carrying nothing is how the clock behaved before the carry existed. On the rows where the anchor is still a real play cursor, the identity then holds against the frames the device will play rather than against the counter, which still counts the discarded ones.

The failed-query row is weaker, and deliberately so. With no cursor to anchor at, the rebase anchors at zero, which is neither cursor, so the identity is left broken and the reported position degrades to the serviced total: it stops following the device between callbacks. That is tolerated because the query only fails on a stream that is already dead, after which the sink pins the position to the last good value and this mapping is never consulted.

A rebase also has to neutralise callback information that has not reached the history yet. The reader counts the frames of every item it takes off the queue, and a rebase records the count it accounted for, so an item landing at or below that point is dropped rather than appended. Without that, an item stranded on the audio thread during the seek would be applied afterwards and, being underrun-only, would merge into the carried chunk and extend the window that advances no media time by the whole stranded silence.

The unplayed audio may be leftover pre-seek audio, silence produced during the seek, or a mixture. Either way it was handed over before the seek and is not part of the new position.

Threading

• The data callback runs on the backend’s realtime thread and must not block or allocate.

• UpdateFrameHistory is called from that callback; GetPosition from the state machine’s thread.

• On macOS the position read is deliberately lock free. Callback information reaches the reader through a single-producer single-consumer queue, applied by AudioClock::ApplyQueuedCallbackInfo, which both GetPosition and Rebase call before touching the history. Anything mutating the history must run on that reader thread and touch only reader-side state; the callback may still be running.

• Elsewhere a mutex guards the history on both sides.