Using voice and images in an existing workflow

OpenAI announced GPT-4o on 13 May as a model that works across text, audio, images and video. The demos are impressive, but I'd start somewhere much more ordinary, with a step in an existing process where typing is slow or loses detail. Someone describing a damaged part, reading a label while their hands are full, finding a field on a printed form, or trying to explain a visual problem in some software. Letting them show it or just say it could shorten that step. I'd pick one moment like that, ask for the specific image or voice input it needs (rather than switching on a camera or microphone stream by default), and leave the rest of the process alone until it proves useful.

Speech gets misheard and people change direction halfway through a sentence. Images can hold several plausible values, or miss the context you'd need to read them. So anything important the model picks up should come back for confirmation before it changes a record. For a spoken address, show the parsed address in editable fields. For a photographed label, put the extracted identifier next to the crop it came from. How much checking you ask for should depend on what happens next. A casual search probably needs none, but an amount, an identity detail, an appointment time or a message going to someone else deserves a proper look. If the model's unsure, ask one focused question.

Then there's what you keep. You could end up with the original audio or images, a transcript, extracted fields, model responses and the user's corrections. I'd decide the record for the task before collecting anything. If a photo supports an asset inspection, you probably need the original image alongside the accepted observation. If voice is just a quicker way to fill a low-risk search box, the corrected text is likely enough. Where a correction changes an operational fact, store what the system proposed, what the user accepted and what happened next. Device permission for the camera also doesn't decide who inside the product can see the material, so the app needs its own access rules. Someone who can upload a photo shouldn't automatically be able to browse every image on the account.

The GPT-4o announcement says access is rolling out in stages, so I'd test only what's actually available in the API or interface you'll use. Build cases with poor lighting, background noise, accents, rotated documents, small text, and someone else's details sitting next to the thing you wanted. Run them on the devices and networks people really use, since upload speed and mobile permission behaviour can matter before model quality does. If someone denies microphone access, there should still be a text path.

I'd measure more than accuracy, too. Time to finish the task, recaptures, corrections and abandoned attempts, compared with the current typed or manual route. A voice flow that transcribes well but is fiddly to correct might end up slower. Keep the model's output as a proposal until a person accepts it, and have downstream services read the confirmed record rather than a chat message.

So that's how I'd bring GPT-4o's voice and image input into an existing workflow: one step where typing is slow or loses detail, with the model's reading confirmed before it changes a record and a clear decision about what gets kept. It's only worth keeping if people finish that step faster or with fewer corrections than the typed route. I'd take one existing form or support step, find the field people struggle most to type, and test voice or an image beside the current method on the devices people actually use.