Spotify's Speech Monitoring Tech Sounds Creepy as Hell

Thanks to the success of curated playlists like its Discover Weekly feature, Spotify is always looking for ways to improve its algorithm. But as Pitchfork reports, its latest move towards that end goes above and beyond: the streaming service has been granted a patent for technology that will use recordings of users’ speech and background noise to determine what kind of music to recommend them.

Spotify filed for the patent back in 2018, but it was approved on Jan. 12, 2021. The filing outlines some potential uses of the technology, including detecting “intonation, stress, rhythm, and the likes of units of speech” from a user’s voice as well as being able to identify their emotional state, gender, accent, age and environment (that’s where the background noise comes into play) based on the audio recording to help curate music for them.

“What is needed is an entirely different approach to collecting taste attributes of a user, particularly one that is rooted in technology so that the above-described human activity (e.g., requiring a user to provide input) is at least partially eliminated and performed more efficiently,” the patent filing reads.

Of course, in addition to raising a slew of privacy concerns, this begs the question: can you actually tell what kind of music someone wants to hear based on their voice? It seems as though any curation that could be done based on gender or accent, for example, would be simply rooted in harmful stereotypes. (Are women going to be fed more female artists than men are? Is someone with a Southern accent only going to get country songs in their curated playlists?) Even something like emotional state seems like a precarious thing to try to base a musical recommendation around; some people like to wallow and listen to sad songs when they’re feeling down, while others prefer to pick themselves up with some upbeat tunes. How can an algorithm possibly know what someone’s personal preferences about that sort of thing are?

We’ll have to wait and find out. As Pitchfork notes, “It’s currently unclear whether Spotify has established a roadmap for the implementation of this tech into its desktop or mobile apps, or what form this implementation might take. It is also unclear if the technology currently exists or if the patent is speculative.”