What a Medical Director Should Ask Before an AI Pilot

A pilot is a clinical decision; ask the questions you would ask of a new protocol.

What should a medical director ask before a clinical AI pilot?

The medical director signs off on the pilot and answers for it at the quality committee. These are the questions that make the sign-off defensible.

The cost of a poorly scoped pilot is not the pilot. It is the next one. A pilot that ends with no result the quality committee can read closes the door on the category for two years. The questions below produce a result the committee can read either way.

Scope: what the tool claims and what it does not

Ask the vendor to finish the sentence "this tool does not" without prompting. A clinical tool has edges.

Gia reaches patients between visits by phone or video, listens, and returns a ranked worklist with reasons. A check-in is a screening conversation. Gia is clinical decision support, not a diagnosis. Gia places no orders and changes no medication.

Ask what happens to a result the clinician disagrees with. The answer should be nothing. The clinician closes it with a note and the record shows the disagreement. A tool that escalates past the clinician is not a co-clinician.

Evidence: per outcome, not per product

A product is not validated. Outcomes are. Ask for the evidence for each outcome you intend to act on.

Scienza's foundation model for aging was tested under a pre-registered sealed evaluation. Five outcomes passed. Falls replicated on a second look. Hospital return is a trained head without a sealed-test result. The manuscript has been submitted to JMIR AI. Use the outcomes that passed. Treat the others as what they are.

Ask how the evidence will be updated and how you will hear about it. A manuscript under review becomes a published paper or a revision. Either way, your committee should hear it from the vendor before it hears it elsewhere.

Workflow: what happens to every output

Trace one result from the call to the chart. Who sees it, in what order, and what can they do.

With Gia the result lands on the clinician's worklist with Proactive Decision Actions. Options, never orders. The clinician picks one, adds a note and assigns follow-up. One click on Initiate Assessment sends Gia to administer the BIMS, a cognitive screening, or an ADL and IADL review in conversation, with results in under two minutes. The decision and the follow-up write back to PointClickCare, Epic or your EHR.

Then trace a result nobody acts on. It should still be in the record, dated, attributed and marked as reviewed. The pilot's value to the committee is as much in what was looked at and set aside as in what was acted on.

Population: who actually gets reached

Ask what the patient needs to have. If the answer is an app, a login or a smartphone, your oldest patients are excluded before the pilot starts.

Gia calls any phone, including a landline, and runs video on a tablet, phone or TV. She speaks 44 languages and answers in the one the patient uses. Missed calls are retried. Caregiver check-ins reach the family member who sees the patient every day.

Ask what happens to the patient who never answers. Gia records each attempt and routes the patient to a nurse follow-up queue after the retries run out. No check-in silently disappears. The patient who could not be reached is a result, and a clinician sees it.

Governance: what you sign and what you can audit

Ask for the business associate agreement before the data flows. Gia operates under HIPAA with a BAA signed with each customer. Results write to your EHR.

Ask what the audit trail holds. Gia's follow-up record keeps every attempt, every result and every clinician decision, dated and attributed. See governance.

Ask where the population analytics run. Gia Health Analytics is hosted by Scienza and sold through the AWS Marketplace and Google Cloud Marketplace. Your records never train the model. Your security team audits it with the tools they already use.

Success criteria, written before day one

Decide what the pilot has to show before it starts. Reach rate. Follow-up completed within the window. Assessments initiated and documented. Clinician decisions recorded. Time your nurses stopped spending on calls.

Do not accept a financial projection as a success criterion. Measure what the pilot can show in your own record.

Where to start

Pick one unit or one discharge cohort, one clinician who owns the worklist, and a 30-day window. Write the success criteria into the pilot agreement.

See how post-acute and senior living teams run Gia, then book a working session to draft the criteria with us.