A video-to-video comparison cannot select a text-to-video route

General
1

CometAPI’s Wan 2.7 API guide separates two questions that are easy to blur when choosing a video route: can an application complete a generation request, and is the output good enough for the user’s task?

The guide describes model ID wan2.7 through CometAPI’s asynchronous POST /v1/videos workflow: create a job, poll for its status, then download the result. It recommends starting with a 5-second, 720p text-to-video request before adding image references or higher resolution.

That is an integration starting point, not evidence that text-to-video is the best choice for a particular product. The same guide cites a Design Arena video-to-video result and explicitly warns against generalizing it to text-to-video or image-to-video leadership. I would not use that result to choose a text-to-video default in a CometAPI-backed application. No current leaderboard rank is established here.

Separate the first successful request from the quality decision

For evaluating this CometAPI route, I would use two checkpoints:

  • Workflow checkpoint: follow the guide’s small text-to-video starting request and confirm the create-poll-download loop. Treat this as an integration check, not a quality comparison.
  • Task checkpoint: evaluate outputs for the mode the application will actually expose. Keep duration, resolution, and input requirements comparable within that task. If the intended feature uses image references, a successful text-only request is not enough to choose that feature’s default.

For the interface, my design recommendation is to distinguish “request submitted” from “result available.” An asynchronous CometAPI job gives users a waiting stage to navigate; a benchmark result does not tell you how to explain that stage or whether the finished clip meets their brief. This is a design proposal, not a measured usability finding.

2

For the image-reference task checkpoint, I would add a shot-level acceptance sheet rather than a single “looks good” score. A clip can preserve the opening composition yet lose the details that made the reference useful.

For example, take a reference of a ribbed glass bottle with a paper label on a stone surface, and request a slow camera move. Before generating, mark which properties must stay fixed: bottle silhouette, label placement, rib spacing, and contact with the surface. Separately mark what may change: reflections, visible background, and perspective as the camera moves. That distinction avoids penalizing a legitimate lighting change as an identity failure.

Then inspect the opening, midpoint, and final frames alongside normal-speed playback. I would record label deformation, disappearing texture, or a sliding contact point separately from whether the requested camera motion happened. Repeat the same brief across several outputs, keeping the reference, duration, and resolution fixed; one attractive clip is not enough to establish consistency.

This is a proposed evaluation fixture, not a reported Wan 2.7 test. For a CometAPI-backed image-to-video feature, it would turn the task checkpoint into a concrete decision: can the route preserve the brief’s non-negotiable visual details while producing the requested motion?