America's phones learned to write this year, and most of us handed over the keys without asking where the car was going.
Apple's new Apple Intelligence and Google's Gemini features both now summarize your texts, draft your replies, and sort your notifications by "importance." That last one sounds harmless until you realize an algorithm is deciding which of your relationships deserves a red badge and which gets buried.
The mechanics are simple enough to ignore.
Your assistant scans incoming messages and flags what it thinks matters.
Your kid's short "ok" sinks to the bottom.
In practice, it's a silent filter between you and everyone you know, shaped by an opaque model that was never elected to anything.
When Samsung and Google rolled out on-device summarization, users quickly noticed missed messages, flattened tone, and drafts that read like HR wrote them.
One wrong summary of a medical result, a landlord's warning, or a breakup text is all it takes to turn a convenience feature into a crisis.
And there's no undo button on a reply you didn't write but did send.
The deeper problem is what we trade for the shortcut.
Attention is the last untaxed resource most Americans still own outright.
Every time an algorithm decides what deserves ours, we practice a smaller version of outsourcing our judgment.
Children growing up with these tools may never develop the muscle for reading a room, a message, or a person.
We're not banning calculators here — we're letting one do the thinking before we've learned the math.
Meanwhile, the data plumbing behind these features is a black box wrapped in a privacy policy.
On-device processing sounds reassuring until you read the fine print about what syncs to the cloud.
Researchers at several universities have already shown that "anonymized" message metadata can be re-identified with troubling ease.
It's to treat AI features the way you'd treat a new roommate: helpful, but not entitled to your mail.
Turn off auto-summaries for anything involving money, health, or family.
Keep the AI for grocery lists and meeting notes.
Read your actual messages from actual people.
It's about noticing that the most intimate interface in your life now has opinions about your life, and those opinions were set by someone you'll never meet, in a room you'll never see.
Final Thoughts
We just got more comfortable being managed by it.