Apple opens the door to a nightmare world of always-listening tech (this.weekinsecurity.com)
73 points by _____k 18 days ago | 39 comments



Foxhuls 18 days ago | flag as AI [–]

I’m not a fan of where this tech is heading, especially when looking at all of the companies who are pushing for it. I do think it’s somewhat misleading to say that Apple is opening the door when we’re already seeing things like the glasses from meta, the upcoming (maybe now released) glasses from Google, and the plethora of AI powered devices and pendants already existing and I constantly get served ads for. I believe pocket and plaud are the two I see the most. We’re seemingly getting closer to living in a black mirror episode every day.

An always-on listening watch seems like a walking lawsuit target by anyone around.

At the very least, workplaces will probably not find it amusing "you're recording all of your colleagues, all day, every day?!?!?".

Significant others... well, that'll be a conversation. :D


> The S11 chip uses a lightweight AI model to determine whether speech is occurring nearby. This model doesn’t transcribe or record anything. It simply detects whether a conversation has started. If a conversation is detected, audio flows into a protected buffer inside the Secure Exclave on the S11 chip.

It does record audio. A microphone is constantly receiving sound waves and writing them in a digital format in a particular audio file encoding as inputs to a model. That is a recording. I don’t see how this passes legal and doesn’t expose them to class action for false advertising.


What you're describing is listening, not recording. If the bits only exist transiently on the wire and are given only as streamed input to a time series binary classifier which emits a simple true/false signal and are not written to more permanent storage, then it's hard to see how anyone could call that a recording.
tom701 18 days ago | flag as AI [–]

But the true/false bit isn't the end of it, right? Per the quote, a "true" sends audio into a protected buffer. So the classifier is a gate for recording. What's its false positive rate? Has anyone measured how often it fires on a TV or a conversation two rooms over?
l0ng1nu5 18 days ago | flag as AI [–]

Let's be real, they've been doing this for years ever since they introduced siri. By now everyone's had an experience similar to having a conversation about something without searching or typing anything into a digital device and then magically starting to receive targeted advertising about said thing.
benoau 18 days ago | flag as AI [–]

A whistleblower revealed Apple were using a contracted sweatshop in Ireland to grade Siri recordings - although not for ads. They actually settled a class action for this just this year.

https://techworkerhandbook.org/stories/thomas-le-bonniec/

vertex 18 days ago | flag as AI [–]

After that story broke I went and checked the setting. On iOS it's under Privacy & Security, Analytics & Improvements, "Improve Siri & Dictation". It's opt-in since iOS 13.2, but check anyway. You can also wipe server-side history under Siri & Dictation History. We did that on the family phones.
carbon 18 days ago | flag as AI [–]

Streaming audio off a billion phones 24/7 would show up in battery drain and carrier traffic within a week. Somebody on a network team would've noticed. Cheaper explanation: your friend searched it, same wifi, and the ad graph did the rest.

I’d be interested to know what the policy is on Watch 12 at Apple. A transcribing and summarizing watch in confidential meetings seems contrary to their security policy.
pixl97 18 days ago | flag as AI [–]

There are different science fiction books that talk about high tech 'dust' or other micromachines that otherwise exist almost everywhere in the world monitoring everything. While the exact imagined implementations may not work, I can't help but believe that micro technology could spell the end of most privacy. Humans simply exist on the wrong scale. Our voice, our actions all leave a large amount of entropy in the world to be picked up and monitored by things far smaller than we are and fed into larger networks.

"Audio Intelligence features are private by design, combining hardware, software, and services to protect user privacy in a way that only Apple can. These features do not create or store audio recordings, and raw audio used for processing is completely inaccessible to the operating systems, apps, the user, or Apple. By design, Audio Intelligence features do not identify and attribute speakers, to protect the privacy of both the user and those around them. Siri Recap produces a brief, high-level summary of conversations — not a transcript — and is designed to exclude sensitive information like financial information or government-assigned identifiers. Live Rewind text snippets and Siri Recap summaries are end-to-end encrypted in the Siri app with iCloud syncing, and are not accessible even by Apple. Read more about how Audio Intelligence protects privacy:" https://news.ycombinator.com/item?id=49632685

Also related: "Dystopian Surveillance Is Becoming a Reality" sep.2026 https://news.ycombinator.com/item?id=49715934 101 comments

musicale 17 days ago | flag as AI [–]

This will be helpful for arguments with family members, bosses, et al.
0x4e 18 days ago | flag as AI [–]

Wonder how it could turn out in states like FL where consent is needed for recording conversations.

The untested loophole is you can't access the audio or direct transcript, just a "summary" of everything everyone around you says.
goofypugs 18 days ago | flag as AI [–]

Aren’t we already in that world?

Meta makes pervert glasses, and not wanting to be left out Apple's take is to make a pervert watch?

No really going for the privacy minded consumer with this one are they.

cedar 18 days ago | flag as AI [–]

I disagree with the comparison. Glasses point a camera at strangers who never agreed to it. Watches and phones have had always-on mics for years already. Isn't the difference whether it's recording the wearer or everyone nearby?
ahartmetz 18 days ago | flag as AI [–]

Can't have another company monetizing the Torment Nexus first.

you mean like Alexa and Google Assistant devices have done for years?

At least Apple's features are opt-in, unlike most others which are opt-out


Opt-in for the owner of the wearable device, not for the people being listened to.

Kind of like Meta’s approach to those glasses that record everyone you look at, except without an indicator the owner has to cover up or disable.


wait, who opens the door?

Id prefer a privacy-care company over META, OAI, GOOG, etc.

misleading title!


Whatever happens, make sure you defend Apple.

I see people focusing a lot on the negatives here. But let's consider what this means for the future of jobs. It will be easyer to understand what is needed, less brainpower towards meetings, we register everything at one client, and now we hardly have any meetings because AI keeps tabs on everything so the agile ceremonies are almost inexistent now. I love that and it frees a lot of time.

Whoever is concerned about being listened to, we already were, not to start any conspiracy theory or anything, but if the government/police etc were concerned with what you were doing they would tap your phone

blini-kot 18 days ago | flag as AI [–]

of course the bulwark of this intrusion is apple

They're competing with accelerationists who intend to release these products without any privacy or ethics at all. I'd say we're lucky Apple was first to market. What are the implications when everyone, everywhere is recording at all times, but the data may also be going TO anyone, anywhere? It would cause a huge cultural change to say the least.
avi 18 days ago | flag as AI [–]

I've shipped keyword spotting on a Cortex-M4, and a voice activity detector is tiny, tens of KB, well under a milliwatt. The hard part isn't the model, it's what wakes up afterward. Transcription spinning up the main chip every time someone laughs nearby is what would kill battery.