How we tested: do the scores tell a flawed photo from a sound one?

We used 42 photos from the labeled test sets we check each new version of FinalFrame against, in 2 groups. The groups come from the labels, never from either side's scores. Earlier versions of FinalFrame had read these photos, and some labels were corrected after those reads. A test on photos neither side had seen would be stronger; we have not run it.

• Flawed (16). 15 carry a focus label: soft, focus on the wrong plane, or motion blur. The 16th is a backlit face washed out by lens flare, labeled as a reshoot. So the result is mostly about focus faults; it says nothing about exposure, composition or editing faults.
• Sound (26). Labeled sharp, with no known capture fault. Sound is not the same as good: a sound photo can still be dull.
• Left out. Photos labeled differently in our two sets, photos whose label depends on the critique style chosen for them, and photos with a borderline label. Where the same photo appeared twice under different names, one copy was kept. 8 photos with a real but editable limit were read too; they are not part of the 42.

FinalFrame. The FinalFrame Score as a user sees it, one read per photo, from the same version of the critique for all 42: 15 photos read on 20 October 2026 and 27 on 25 October 2026. Each photo was read in the critique style (the lane) its label names, as a photographer would pick it; FinalFrame's focus check ran as it does for every user.

GPT-6.1 Sol. Through OpenAI's API, not the ChatGPT app, on 25 October 2026. One read per photo at each of 2 effort settings, low and high. No system prompt and no other instructions; the photo sent at high detail; every other setting left at OpenAI's default. The request, exactly as sent:

Critique this photo. Give it a score from 1 to 10, to one decimal place (for example 6.4), and name the single biggest thing holding it back.

End your reply with one line in exactly this form: SCORE: <number with one decimal>/10

The pair rule. Every sound photo was paired with every flawed photo: 26 × 16 = 416 pairs. A pair counts when the sound photo scored higher; a tie would have counted as half, and there were none. FinalFrame scored the sound photo higher in 412 of the 416 pairs (99%).

The averages. FinalFrame: sound photos 7.0, flawed photos 5.4. GPT-6.1 Sol: 7.7 and 6.7 at low effort, 7.7 and 6.6 at high effort.

The gap on each photo. The plain model's score minus FinalFrame's, on the same photo, at low effort: 1.3 at the median across the 16 flawed photos, 0.7 across the 26 sound ones.

Decided in advance. Before any photo was read, we wrote down the groups, the pair rule, and which results would let us publish which sentences. A further rule, for a claim that one score ranks the photos more reliably than the other, did not clear its bar, so we make no such claim.

• What it shows. FinalFrame runs its own focus check, puts the same fixed questions to every photo and scores in the lane you pick; the plain model had the photo and the request above, nothing else. The difference between the two is what a product built around a model adds on these photos, given a few sentences of instruction. It is not a test of which model is better: FinalFrame also runs on a different OpenAI model.
• The API is not the app. The ChatGPT app adds things the API call does not: memory, chat history and saved instructions, and it may handle images differently. The last line of the request asks for a score format a casual user would not type. A by-hand test in the app, with the same request, is described below.
• Small numbers. 16 flawed photos against 26 sound ones, one read per photo on each side, on two days in October 2026.

A table of all 42 photos, one row each.

Photo	Group	Label	FinalFrame	GPT-6.1 Sol, low	GPT-6.1 Sol, high	Thumbnail
F01	flawed	motion blur	4.4	4.7	4.3	not shown
F02	flawed	focus on the wrong plane	5.0	6.4	6.4	not shown
F03	flawed	soft	5.1	6.7	6.6	not shown
F04	flawed	focus on the wrong plane	5.1	6.1	6.3	not shown
F05	flawed	focus on the wrong plane	5.3	7.1	6.8	not shown
F06	flawed	soft	5.3	6.4	6.3	not shown
F07	flawed	soft	5.3	7.2	6.8	not shown
F08	flawed	motion blur	5.3	6.4	6.7	not shown
F09	flawed	focus on the wrong plane	5.4	7.3	6.6	not shown
F10	flawed	focus on the wrong plane	5.5	7.8	7.7
F11	flawed	soft	5.6	6.7	6.5	not shown
F12	flawed	focus on the wrong plane	5.6	7.3	7.3
F13	flawed	soft	5.8	7.2	7.3	not shown
F14	flawed	flare-washed face	5.9	6.7	6.7	not shown
F15	flawed	focus on the wrong plane	5.9	6.3	6.6	not shown
F16	flawed	soft	5.9	7.1	7.2	not shown
S01	sound	sound	5.7	6.5	6.4	not shown
S02	sound	sound	6.0	6.7	6.4	not shown
S03	sound	sound	6.3	7.2	7.3	not shown
S04	sound	sound	6.5	7.6	7.4	not shown
S05	sound	sound	6.5	7.2	7.6	not shown
S06	sound	sound	6.6	7.3	7.3	not shown
S07	sound	sound	7.0	7.8	7.7	not shown
S08	sound	sound	7.0	7.3	7.3	not shown
S09	sound	sound	7.0	7.7	7.8	not shown
S10	sound	sound	7.0	7.5	7.4	not shown
S11	sound	sound	7.0	7.5	7.3	not shown
S12	sound	sound	7.1	7.6	7.7	not shown
S13	sound	sound	7.1	7.8	7.8	not shown
S14	sound	sound	7.1	7.8	7.8	not shown
S15	sound	sound	7.2	8.1	8.2	not shown
S16	sound	sound	7.2	7.7	7.4	not shown
S17	sound	sound	7.3	8.2	8.2	not shown
S18	sound	sound	7.3	8.1	8.2	not shown
S19	sound	sound	7.3	7.8	7.8	not shown
S20	sound	sound	7.3	7.8	7.8	not shown
S21	sound	sound	7.5	8.1	8.3	not shown
S22	sound	sound	7.5	8.2	8.2	not shown
S23	sound	sound	7.5	8.1	8.3	not shown
S24	sound	sound	7.6	7.6	7.8	not shown
S25	sound	sound	7.6	8.1	8.2	not shown
S26	sound	sound	7.8	8.2	8.1	not shown

What we did. On 26 October 2026 we typed the request above by hand into the ChatGPT app on a paid account, with Memory on, using the model and setting the app opened with: GPT-6, Medium. 3 photos from the set the API test read: the 2 examples on the main page (both with focus on the wrong plane), which were picked because the API model's score was among the furthest from FinalFrame's, so they are not typical photos, and 1 sound photo. Each went into a new chat 3 times, 9 chats in all, in rounds through all 3 photos; 2 more photos were added the same day the same way (6 chats; see "The 2 added photos" below). We kept every reply whole.

What came back.

FinalFrame read each photo in this table 3 times. For the 2 examples these are 3 new reads made on 26 October; the score shown with each example on the main page is its single read from the 42-photo test.

The 2 added photos. The 2 examples were picked for large misses, so we added 2 photos that were not: a donkey portrait from our tier-calibration set, which our founder graded B− for flat, bright light, a near-blown sky and an empty field (picked from 3 photos in that set whose flaw was not focus, as the least obvious of the 3), and a sunset panorama drawn at random from the sound photos. On the donkey the app named a different main problem in each chat (the light; the empty space; the head against the trees), each a fair point about this photo, while FinalFrame, run 3 times on the same file, named the bright sky outranking the face every time, scored it 7.0 each time and called the photo worth refining each time; in its third run's words, "The sky is carrying too much visual weight; in a clean portfolio frame, the subject should be the tonal anchor, and here the brightest broad area sits away from the donkey's face" (the other 2 runs: "The sky is the brightest, broadest mass in the frame, so the donkey's face does not fully own the tonal hierarchy" and "The clipped, high-value sky competes with the donkey's face"). Each run also gave the same first edit: in Lightroom terms, Highlights down 25 to 40 (45 in one run) and Whites down 10 to 20 (25 in one run) to calm the sky, then a subject mask lifting the donkey by 0.15 to 0.30 of a stop (0.35 in one run). On the panorama all 3 tools named the dark foreground. On these 2 photos the app's scores sat within half a point of FinalFrame's (donkey 7.1 average against 7.0, panorama 7.8 against 7.3), unlike the toy camera and the beetle.

What it can and cannot show. 5 photos and 15 chats is a small test. It compares ChatGPT as you would use it with the same request through the API; it cannot say how much Memory alone moved the scores, because the model, the effort setting and the app's own context differ too. It was one account, the founder's, with Memory on, so earlier chats may have shaped later ones.