Skip to writing
Skip to content
← All writing
Cradle / 4 minute read / Draft

Cradle: waiting for a second frame

What a native overlay should do when its understanding comes from pixels.

Try example

Cradle began around Marathon and uses native Swift, OCR and visual parsing to work with what is on screen. Unlike an application reading its own structured data, an overlay has to infer which interface it is looking at. A fleeting frame can be convincing and still be wrong.

The overlay follows supported remote-play windows and displays inventory or trade information. Its capture code also has a main-display fallback when no supported window is found.

Recognizing a screen is not yet showing a chip

OverlayPresentationGate separates recognition from presentation. Without the explicit-index override, it waits for two matching observations to return show and two missing observations to return hide. A single observation returns hold, preserving the visible state.

Interactive / Cradle

Two matching frames

Feed the overlay a matching or missing frame and observe its decision.

  1. First match: hold
  2. Second match: show
  3. First miss: hold
  4. Second miss: hide
Behavioral port of OverlayPresentationGate.observe. Synthetic matching/missing inputs; no OCR runs here. The explicitIndex override and separate missing-window counter are outside this example.
CradleOverlayCore/ScreenClassifier.swift · OverlayPresentationGate.observe excerptSwift
if hasState {
    matchingFrames += 1
    missingFrames = 0
    return explicitIndex || matchingFrames >= 2 ? .show : .hold
}

missingFrames += 1
matchingFrames = 0
return explicitIndex || missingFrames >= 2 ? .hide : .hold

Give uncertainty a place in the pipeline

Cradle visual-to-overlay pipeline
  1. Capture window
  2. OCR / visual parsing
  3. Recognized screen state
  4. Presentation gate
  5. Native overlay

The gate is small enough to understand independently of the recognizer. Feed it match, miss, match: the overlay stays hidden. Feed it match, match: it can appear. Once visible, one missing observation leaves it in place. That temporal behavior reduces flicker without pretending the recognizer is certain.

Stability has a cost

Waiting for confirmation introduces delay. The README describes roughly 0.7-second sampling, so presentation timing depends on when the screen changed relative to capture. Two matching frames also do not establish semantic correctness: a consistently wrong recognizer can still pass the gate.

A missing source window and the explicit-index override take separate paths. This example covers ordinary matching and missing observations. It does not run capture or OCR.

What you can inspect nowFeed the presentation gate

Try matching and missing frames in the behavioral port. It establishes the gate's hold/show/hide logic, not OCR or the native overlay.

Continue readingExplorer: from discovery to action