Skip to content
CLEARER IDEAS. BETTER EXECUTION.PROGRAMMING · CREATIVE AI · WORKFLOWS
Menu & topics
Programming

A Media Job Is Still Running: Bound Polling With States and a Deadline

Build a small synchronous polling adapter with explicit queued, running and terminal states, a deadline and call limit, tested with twelve fake-clock fixtures.

A stopwatch, external drive and two blank cards beside a laptop and ceramic coffee cup near a window
A stopwatch, external drive and two blank cards beside a laptop and ceramic coffee cup near a window.
The short answerA nonempty response does not mean a media job is done. This local adapter checks the expected job ID and an explicit state, stops at a deadline or call limit, and accepts a done result only with a nonempty artifact reference. The tests use synthetic responses and a manual clock; they generate no video and make no provider request.

Evaluating bool({"id": "j", "state": "running"}) returns True because a nonempty dictionary is truthy in Python. The first test reproduces that evaluation; it does not run a broken network loop or observe a rendering service. If a caller used that boolean as its completion condition, it could advance while the reported state is still running. Our replacement reads an explicit local state instead. The fetch callable in this experiment returns synthetic dictionaries, so its return establishes only that the callable produced an object. It provides no evidence about transport success, a remote workload, or a generated asset. That small distinction determines what the caller is allowed to do with the next value.

The response arrived, but the job is not done

The broken_done helper in the companion module illustrates how truth value testing fails in media processing workflows. Passing the dictionary {"id": "j", "state": "running"} into broken_done returns True, yet the work described by that dictionary remains incomplete. If an application relies on such an evaluation to break out of a polling loop, subsequent code will immediately attempt to read downstream artifact keys that have not yet been produced. In our isolated tests, the fetcher returns synthetic in-memory state records, meaning no socket activity or provider transport is exercised at all. Relying on truthiness treats the mere presence of data as a signal of finished execution, collapsing distinct lifecycle phases into a premature and invalid exit condition.

The local contract treats queued and running as two responses that require another poll. It does not define what computation happens in either state. In particular, the fixture does not observe queue allocation, frame encoding, audio mixing or progress percentages. The adapter also does not enforce a transition graph: for example, a running response followed by queued would still take the waiting branch, provided the other checks pass. That sequence was not included in the twelve tests. Before adding a real service, decide whether its states can move backwards and translate them deliberately. A familiar state label is useful vocabulary, but its operational meaning must come from the service contract you actually connect.

Each response must carry the expected identifier. The wrong-identity fixture calls poll for j and supplies a dictionary with id set to other; the test expects ValueError. The implementation rejects that mismatch before inspecting its state. This check compares supplied strings. It does not authenticate the response, prove that a remote service performed the named operation, or verify that an artifact belongs to its claimed job. Keep those separate responsibilities when adapting an external client. Passing an ID check means this local record meets an association rule, while the trustworthiness of the record depends on how fetch obtained it and on the surrounding application contract.

Define the local state contract before polling

Python documents time.monotonic as a clock that cannot go backwards and whose reference point is unspecified; differences between readings are meaningful. The example injects a manual clock rather than measuring wall-clock performance. Python3.12 time: monotonic clock

Apply artifact file acceptance after obtaining a reference and downloading bytes through a separately defined client. Polling a job state does not establish that its artifact is a playable video or even that the reference can be fetched.

Parameter validation forms the initial perimeter of the poll function before any iteration takes place. The adapter checks that job_id is an instance of str and is not empty, raising ValueError("Expected a nonempty job ID") otherwise. It further validates that timeout and interval are positive finite numbers using math.isfinite. Because Python booleans are instances of int, boolean literals passed as timeout or interval are currently evaluated as numeric ones or zeros rather than rejected outright, while passing string arguments triggers a TypeError during math.isfinite evaluation. The max_calls parameter is checked with type(max_calls) is not int or max_calls <= 0, which specifically excludes booleans. These checks establish boundary conditions for caller input.

Once inside the loop, the adapter validates the structure of each response before evaluating its operational state. If the returned object is not a dictionary, or if response.get("id") does not strictly equal the requested job_id string, the function raises a ValueError indicating an identity mismatch. If the identity matches, the adapter examines response.get("state"). The local contract accepts exactly five string values: "queued", "running", "done", "failed", and "cancelled". Any other value, such as an undocumented vendor status like "processing" or an integer code, causes the adapter to raise ValueError("Unknown job state"). This rigid schema ensures that unhandled remote states halt the loop immediately rather than idling indefinitely.

When the adapter encounters state == "done", it enforces an additional structural requirement before returning to the caller. The response dictionary must contain an "artifact" key whose value is a non-empty string. If response.get("artifact") is missing, None, or an empty string, the function raises ValueError("Done requires an artifact reference"). Because the check only verifies that artifact is a non-empty string, an arbitrary string consisting solely of whitespace or an opaque file handle like "asset-1" satisfies the condition. The adapter performs no network fetching, URL parsing, byte validation, or media container verification on this reference; it merely confirms that an opaque identifier was provided alongside the terminal state.

The failed and cancelled tokens both raise JobFailed(state), a RuntimeError subclass. The state string remains available in the exception arguments, although the recorded tests assert the exception class rather than inspecting those arguments. This is the local decision to stop polling when either token is reported. It does not send a cancellation request, remove a partial artifact, or establish that no external work remains. The deadline check takes precedence: a late response reaches the timeout branch before this state handling. A caller that wants to reconcile a reported failure with later observations needs a separate policy; this function does not decide whether a remote condition is permanently unrecoverable.

job_poll.py

import math

class JobFailed(RuntimeError):
    pass

def broken_done(response):
    return bool(response)

def poll(job_id, fetch, clock, sleep, timeout=10, interval=1, max_calls=20):
    if not isinstance(job_id, str) or not job_id:
        raise ValueError('Expected a nonempty job ID')
    if not math.isfinite(timeout) or timeout <= 0 or not math.isfinite(interval) or interval <= 0:
        raise ValueError('Positive finite timing values are required')
    if type(max_calls) is not int or max_calls <= 0:
        raise ValueError('Positive integer call limit required')
    deadline = clock() + timeout
    for _ in range(max_calls):
        if clock() >= deadline:
            raise TimeoutError('Polling deadline reached')
        response = fetch(job_id)
        if clock() >= deadline:
            raise TimeoutError('Polling deadline reached after fetch')
        if not isinstance(response, dict) or response.get('id') != job_id:
            raise ValueError('Response identity mismatch')
        state = response.get('state')
        if state == 'done':
            artifact = response.get('artifact')
            if not isinstance(artifact, str) or not artifact:
                raise ValueError('Done requires an artifact reference')
            return artifact
        if state in ('failed', 'cancelled'):
            raise JobFailed(state)
        if state not in ('queued', 'running'):
            raise ValueError('Unknown job state')
        sleep(min(interval, max(0, deadline-clock())))
    raise TimeoutError('Polling call limit reached')

Recorded local check: Twelve unittest methods using a manual fake clock and synthetic state dictionaries. No real sleep, network request, provider call or video generation, no footage or timing benchmark. poll is synchronous and cannot preempt a blocking fetch or interrupt a remote job; after-fetch deadline only detects an overrun once fetch returns. Clock/callables are trusted. Artifact reference is an opaque nonempty string, not a validated/downloaded URL or a playable video. State names form our local adapter contract, not a documented provider schema. Fetch exceptions propagate without retry.; environment: Python3.12.3 on Windows

Follow queued, running and done with a fake clock

The Clock test helper owns an integer t and a list of requested delays. Its now method returns t. Its sleep method first appends the delay to that list, then adds the delay to t. No system sleep or media work occurs. This lets the fixture describe exactly when the next deadline check runs without waiting for that amount of real time. It remains a cooperative simulation: the adapter trusts these callables, and the test does not examine operating system scheduling or a monotonic clock implementation. Use the delay list to inspect the requested cadence, while keeping real client timing as an unperformed integration concern.

The test_queued_running_done method directly exercises the happy path of the state machine across multiple simulated time steps. The test configures an in-memory fetcher with a sequence of three synthetic dictionaries: {"id": "j", "state": "queued"}, followed by {"id": "j", "state": "running"}, and ending with {"id": "j", "state": "done", "artifact": "asset-1"}. The poll function is invoked with job_id="j", default timeout=10, and default interval=1. At simulated timestamp t=0, the adapter fetches the first dictionary, observes the queued state, and executes c.sleep(1), advancing the clock to t=1. It then records the delay of 1 in the delays list before proceeding to the next iteration.

During the second iteration at simulated timestamp t=1, the fetcher returns {"id": "j", "state": "running"}. The adapter checks that the job identifier matches, recognizes the active running state, and calculates the next sleep duration. Because the deadline is 10 and the current clock is 1, remaining time is 9, so min(1, 9) yields a sleep delay of 1. Invoking c.sleep(1) advances the fake clock to t=2 and appends a second entry of 1 to c.delays. At t=2, the third fetch returns the terminal record containing state "done" and artifact "asset-1". The adapter extracts the artifact string and returns it immediately without executing another sleep call.

The test harness verifies this execution by asserting self.assertEqual(result, "asset-1") and self.assertEqual(c.delays, [1, 1]). These assertions confirm that the function returned the exact artifact reference provided in the final payload and that it paused exactly twice for one unit each time. No actual sleep was executed, no network frames were transmitted, and no media files were rendered or played back during this check. The synchronous function trusted the injected clock and sleep callables completely, demonstrating how a deterministic test fixture can verify state progression and interval scheduling without depending on variable system clock behavior or real-time pauses.

Decoupling time from system clocks also exposes the strict operational boundaries of synchronous code. The poll function cannot preempt an injected callable; it can only evaluate the clock before calling fetch, after fetch returns, and when calculating the delay passed to sleep. If an injected fetch or sleep callable were to block indefinitely, poll would remain frozen inside that call because it possesses no background thread or asynchronous cancellation mechanism. The manual fake clock confirms that state transitions and delay calculations follow our specification under cooperative execution, but it does not test operating system thread scheduling, concurrent execution, or uncooperative blocking calls.

Place deadline checks around the fetch boundary

A common design flaw in polling loops is checking deadlines only once at the beginning of each iteration. In synchronous architectures where I/O operations can take significant time, placing a timeout check exclusively at the loop header creates a major blind spot. If an external fetch call takes five seconds to return when only one second remains on the deadline budget, waiting until the next iteration to evaluate time means the application processes an outdated response long after its temporal window has closed. To prevent this, the adapter places clock evaluations both immediately before invoking fetch and immediately after fetch returns control back to the function.

The pre-fetch check evaluates if clock() >= deadline. Because it uses greater-than-or-equal comparison, reaching the exact deadline value causes the function to reject further retrieval and raise TimeoutError("Polling deadline reached"). This prevents dispatching an external call when there is zero remaining time budget. The post-fetch check performs an identical evaluation: if clock() >= deadline after fetch returns, the adapter raises TimeoutError("Polling deadline reached after fetch"). This post-fetch check runs before response identity or state values are evaluated, ensuring that if a slow fetch overshoots the deadline, any returned payload—even an otherwise valid done state—is discarded in favor of reporting the timeout.

This boundary sandwich is verified by test_deadline and test_fetch_overrun. In test_deadline, poll runs with timeout=2 and an iterator returning {"id": "j", "state": "running"}. At t=0, the first poll succeeds and sleeps for 1, setting t=1. At t=1, the second poll succeeds and sleeps for 1, advancing the clock to t=2. At the start of the third iteration, clock() returns 2, which equals the deadline of 2. Because 2 >= 2 evaluates to True, the pre-fetch check immediately raises TimeoutError. In test_fetch_overrun, the fetch callable internally mutates c.t to 11 against a deadline of 10 while returning a valid done payload. The post-fetch check detects 11 >= 10 and immediately raises TimeoutError("Polling deadline reached after fetch").

When scheduling delays, the adapter computes sleep(min(interval, max(0, deadline – clock()))). This calculation clamps the requested sleep interval so that it does not exceed the remaining time before the deadline. However, because our test suite utilizes a synthetic fake clock where c.sleep simply increments an integer, real sleep overshoot was not tested. In a production environment with a real operating system clock, time.sleep can sleep longer than requested due to scheduler latency. Furthermore, the synchronous after-fetch check can only detect an overrun after fetch has already returned control; it cannot preempt, interrupt, or cancel a blocking fetch while that callable is actively executing.

Keep terminal failures separate from unknown states

The test_failed and test_cancelled methods each supply one synthetic dictionary and assert that JobFailed is raised. Neither test checks for a cleanup request, a saved recovery record or an artifact download, because none of those operations exists in the adapter. Likewise, neither fixture proves that every service uses these two labels or considers them permanent outcomes. The useful regression condition is smaller: these known tokens must not fall through into the active waiting branch or return an artifact. If an application later distinguishes user cancellation from worker failure, it can add a richer outcome contract and tests while preserving the current behavior as an explicit compatibility decision.

Nonempty dictionaries are truthy in Python. That language-level property does not interpret the application state inside a response. Python3.12 truth value testing

In contrast, when the response contains a state string that falls outside the defined set of five tokens, the adapter raises ValueError("Unknown job state"). This behavior is tested in test_unknown_state using the payload {"id": "j", "state": "processing"}. Although "processing" might be a valid status in some third-party media APIs, it is not part of our local contract. Instead of defaulting to an active waiting state or assuming failure, the adapter raises ValueError. This fail-fast approach prevents the adapter from silently misinterpreting upstream API changes, schema drift, or mistyped status fields as valid continuation signals, surfacing contract violations immediately during execution.

Order of precedence inside the loop is strictly defined. The post-fetch deadline check occurs before identity validation and state inspection. If a fetch callable overshoots the deadline and simultaneously returns a failed state or an unknown state, the adapter raises TimeoutError rather than JobFailed or ValueError. Similarly, identity checking precedes state checking; a response with {"id": "wrong", "state": "failed"} raises ValueError("Response identity mismatch") rather than JobFailed. In test_done_without_reference, {"id": "j", "state": "done"} omits the artifact key, raising ValueError("Done requires an artifact reference") because the completion contract requires both the done token and a non-empty string reference.

Bound calls even when the clock does not advance

Relying entirely on clock advancement to terminate a polling loop leaves systems vulnerable to infinite loops under specific edge conditions. In automated testing environments, test doubles might provide a static or misconfigured clock callable that returns a constant timestamp on every invocation. Similarly, if an adapter runs in an environment where time increments by zero or where an interval calculation resolves to zero, a loop governed solely by a deadline check could issue unbounded network requests. To prevent resource exhaustion, our implementation enforces a hard limit on the total number of fetch attempts via the max_calls parameter.

The range(max_calls) loop permits at most that many fetch invocations during this call to poll. The limit does not interrupt a callable that has already started. If clock, fetch or sleep never returns, the loop cannot reach the next iteration or the final exception. When all iterations return normally with active states, the function eventually raises TimeoutError with the call-limit message. This bounds the number of loop attempts under cooperative execution; it is not an absolute wall-clock ceiling. That distinction remains necessary even with a positive interval and a monotonic clock, because checking time and cancelling an operation are different mechanisms.

The test_frozen_clock_call_limit fixture directly verifies this behavior. In this test, poll is invoked with job_id="j", a static clock callable returning constant 0 (lambda: 0), a no-op sleep callable (lambda _: None), and max_calls=2. The fetch callable returns {"id": "j", "state": "running"} on every call. Because clock() continuously returns 0, the deadline of 0 + 10 = 10 is never reached by time checks alone. However, after executing the second fetch call, the loop terminates and raises TimeoutError matching the regex "call limit". The test verifies the exception text, confirming termination without relying on a separate call counter observer.

One notable implementation detail is that the adapter executes a sleep call at the end of every active iteration, even after the final permitted fetch attempt before the loop concludes. For example, if max_calls=2 and both fetch calls return running states, the function invokes sleep after the second fetch before the range loop terminates and raises the call limit timeout. In our fake-clock test, the sleep callable is a no-op, so no delay occurs. In a production setting with a real sleep callable, this results in an unnecessary wait after the last call before raising TimeoutError. Callers should be aware of this extra delay when configuring call limits.

Choose the two bounds for different questions. The deadline asks whether a returned observation is still within the polling window; the call limit asks how many attempts this invocation may make. Neither bound establishes service rate-limit compliance or interrupts a stalled transport. Both may be needed when testing a proposed client, but they leave resource ownership and transport timeouts to another layer. Include clock itself among the trusted dependencies, rather than describing every broken temporal environment as handled. The recorded frozen-clock case is one specific cooperative fixture with a no-op sleep, and the after-fetch case detects lateness only after the fetch callable returns.

Inspect the twelve recorded tests and their limits

The verified local test suite consists of exactly twelve unittest methods executed under Python 3.12.3 on Windows. All tests operate strictly against in-memory callables and synthetic state dictionaries. The suite does not make real network requests, issue provider API calls, perform system sleep operations, or generate video files. Elapsed suite execution time recorded in test outputs represents local test runner overhead rather than a performance benchmark. Furthermore, the test suite exercises specific branches defined in our test file rather than every conceivable edge case, and it does not establish thread safety, resilience under concurrent load, or behavior with uncooperative blocking callables.

Five methods establish the basic response behavior. test_reproduce_truthy_running asserts that broken_done returns a truthy value for the running dictionary. test_queued_running_done checks the returned reference asset-1 and the delay list [1, 1]. test_failed and test_cancelled each assert JobFailed. test_wrong_identity asserts ValueError for a response naming other instead of j. The latter three tests check exception classes, not the complete message or exception arguments. Report those assertions precisely: they provide regression evidence for the selected branches, without certifying every possible malformed dictionary, a provider schema or the origin of the response.

Four further methods examine state requirements and time checks. test_unknown_state expects ValueError for processing, and test_done_without_reference expects ValueError when a done response omits artifact. These tests do not assert the full error messages. test_deadline supplies four available running responses with timeout=2; only two are consumed before the clock reaches 2 and the next pre-fetch check raises an exception whose message matches deadline. test_fetch_overrun moves the clock to 11 during fetch against deadline 10, then checks that the raised timeout message matches after fetch. Its done payload is deliberately not accepted as a late success.

The final three tests cover iteration bounds, timing parameters, and unhandled exceptions. test_frozen_clock_call_limit asserts that a static zero clock with max_calls=2 raises TimeoutError("Polling call limit reached"). test_invalid_timing runs a parameterized subtest over timing values [0, -1, float("inf"), float("nan")], confirming that non-positive or non-finite values raise ValueError. However, separate invalid configurations for job_id, interval, and max_calls are implemented in code but not independently tested in this suite. Finally, test_fetch_exception_propagates confirms that an injected OSError raised by fetch propagates directly out of poll without being swallowed or retried.

Attach a real client only after choosing its transport policy

Connecting the local poll adapter to an external media provider requires defining a concrete transport layer inside the fetch callable. In our unit tests, fetch is an in-memory lambda that returns synthetic dictionaries without network I/O. In a production integration, that callable would typically wrap an HTTP client like httpx or requests. Because our adapter contains no network logic, the enclosing fetch callable must be responsible for constructing URLs, attaching authentication tokens, managing connection pooling, and establishing low-level socket timeouts. The adapter treats fetch as an opaque black box, relying entirely on the caller to manage transport-level concerns.

A proposed transport wrapper needs its own timeout policy. The current fetch signature receives only job_id; poll does not pass its internal deadline or remaining budget to that callable. Sharing a deadline through a closure or redesigning the interface would therefore be an additional change with new tests. Even setting an HTTP client timeout to a remaining duration should not be presented as a proven total request deadline: libraries may apply timeouts to separate phases. Define the behavior you need, read that client’s documentation and test it under the relevant failure conditions. This example demonstrates a returned-response deadline check, without implementing transport interruption.

External media APIs often utilize diverse status naming conventions that differ from our local five-state contract. For example, a provider might return HTTP 200 with {"status": "IN_PROGRESS"} or {"code": 202, "phase": "rendering"}. Passing such payloads directly into our adapter would trigger ValueError("Response identity mismatch") or ValueError("Unknown job state"). A production fetch callable must therefore act as an anti-corruption layer, mapping vendor status codes and response fields into our local dictionary format containing the expected "id", "state", and "artifact" keys before returning the dictionary to the polling loop.

Any exception raised by fetch escapes poll. The OSError fixture establishes propagation of that exception class, without declaring that a real transport failure is permanent. A surrounding caller could choose to report the error, resume observation of the same existing job, or use a bounded read-retry policy. An inner fetch wrapper is another possible place for such a policy, provided its waits and attempts participate in a shared budget. None of these alternatives is implemented here. In particular, an observation error is not permission to submit a replacement generation job: remote creation and polling an existing operation need separate reconciliation rules.

The returned string is an accepted reference from the local response contract. In this experiment it is asset-1, not a URL or a generated video. A later integration can resolve the reference, retrieve bytes and apply an artifact acceptance policy before exposing the result to a user. A nonempty string, an HTTP success, or a MIME label alone does not establish that those bytes are usable media. The linked acceptance article describes another stage with its own narrow evidence. Keep that stage separate from this state adapter, so a test of delay scheduling is never reported as a successful download, playback experiment or content-quality assessment.

Prepared with AI assistance and automated editorial reviews. The original Python example was checked locally with synthetic data; no media provider or production workload was tested.