- Home
- AI development
- Multimodal AI
Multimodal AI
Multimodal AI that reads photos, voice notes and scanned paperwork
A multimodal model takes in more than text: a photo of a damaged part, a voice note from a job site, a scanned PDF with stamps. We build the system around it, so the model pulls out the facts you need, puts them in your database or ticket system, and sends anything it is unsure about to a person.
- Prices published on the site
- First working version in two weeks
- Support month to month, cancel any time
One model that reads several kinds of input
Models such as Gemini and Claude read images and PDFs next to text, and Gemini also takes audio and video. That removes a step many offices still do by hand: a person looks at a photo or a scan and types what it shows into a system.
Around the model we build a fixed list of fields to extract, a confidence check on each one, a review screen for the doubtful ones and a connection to your system. Our own example of voice is FirstReply, our AI receptionist on (916) 618-0502: it turns speech into text, answers, speaks back, and keeps a written transcript of every call.
Five kinds of input, and where each one ends up
| Input | What the model pulls out | Where it lands |
|---|---|---|
| A photo of a damaged part or job site | Part type, visible damage, location, a short description, and a label or serial number when it is in frame | A work order or ticket, with the photo attached |
| A voice note from the field | A written transcript, then the job, address, parts used and next steps | The job record in your system, with the audio kept beside it |
| A scanned PDF with stamps and handwriting | Printed fields, stamps, dates and handwritten entries, each checked against format rules | A database row or spreadsheet, plus a review list for the doubtful fields |
| A screenshot of an error | The error text, the app and version on screen, and what the user was doing | A support ticket with a category and a draft reply |
| A short video clip | A written summary and a few key frames, for example the state of a repair | An attachment on the record with the summary, for a person to watch when it matters |
Video is the most expensive input and the least reliable. Handwriting accuracy varies from writer to writer. Fields that fail a check go to a person before they reach the record.
What we put around the model
A fixed set of fields
We agree with you which fields to extract. The model returns only those, in a fixed format, and is instructed to write "not found" instead of guessing.
Checks on every field
Each field is checked: format rules, totals that must add up, a second reading that must agree. Fields that fail go to a review list where a person corrects them with the original image beside the data.
An accuracy test on your files
Before launch we run 20 to 50 of your past examples through the system and report how often each field came out right.
A hand-off to your system
The record lands in your database, ticket system or spreadsheet with the original file attached, so anyone can check it later.
When reading images and audio with AI pays off, and when it does not
A good fit when
- People retype data from photos, scans or voice notes every day
- The inputs are alike enough that you can name the 10 to 20 fields you need
- A wrong field is caught by a quick review and does not cause harm
Something simpler will do when
- The PDFs are digital with a fixed layout: plain text extraction is exact and costs less than a model
- You need exact handwriting, such as a legal form or a signed statement: a person should read and type it
- Video would be the main input: start with photos, and add a short clip only where a photo cannot show the problem
Priced by input type and target system
One workflow running end to end, or one internal tool.
Several systems made to work as one: integrations, roles, reports.
One input type into one system, for example scanned intake forms into a database with a review screen, usually lands in one process. Several input types, or one input feeding several systems, land in connected systems. Model usage depends on volume and on the input, with video the highest. We estimate it in the audit, and the bill goes to your own account.
Care $300 to $500 a month. Active $750 to $1,500 a month. AI model usage, GPU time and hardware are paid to the provider, on accounts in your name.
Questions about multimodal AI
How accurate is it on handwriting?
It varies by writer and by form. Printed text on a clean scan comes out far better than a cursive note on a crumpled page. We test on 20 to 50 of your own forms and give you the result per field, and anything below the threshold goes to a person.
Can it work from a voice note or a phone call?
Yes. The audio is turned into a transcript, and the facts are pulled from the transcript into your record. Background noise and strong accents lower accuracy, which the test on your own recordings shows. FirstReply works this way on live calls, and AI for calls and chat covers the phone side.
What happens with a blurry or unreadable image?
It is marked as unreadable and sent to a person with a note about what is missing. The model is instructed not to guess, and the field checks catch values that do not fit. For field photos we also give workers two rules: hold the phone flat and keep the label in the light.
Is video practical?
Rarely as the main input. It takes the most processing of the five and is the hardest to check. When a photo cannot show the problem, we use a short clip and have the model write a summary and pick key frames, and a person checks them.
Where do our images and recordings go?
To the AI model the system calls, and we tell you which one and what it sees before we build. Where the files cannot leave your network, an open-weight model on your own server can read them. Accounts are opened in your company's name.
Do we need special cameras or scanners?
No. Phone photos and an office scanner work. We write a one-page guide for your staff on how to take the photo, because a flat, well-lit picture helps accuracy at no cost.
Related services
- Computer vision Read photos and scanned documents
- Language processing (NLP) Sort, extract and summarize text
- AI for calls and chat Answers calls and books the appointment
- FirstReply Our AI receptionist, live on our own line
- AI agents Agents that finish a task, with your sign-off
- AI development Every AI service we build, on one page
Tell us what's slowing you down.
You get a free written review by email within two business days.