Digging My Way Out of an Algorithmic Pit

Or, How to Convince Spotify I Have Free Will

I wrote last week that one of the questions I constantly ask is “what is it all for”? Another one is “how do we forge a satisfying and authentic self” in a world where what we read, watch, buy, and listen to are increasingly conditioned and determined by corporate interests? I love technology and I love convenience—but do I love them so much that I’m willing to sell my soul? That’s the question I’m asking today, relative to music, but the more I think about it, the more I recognize that this question lies at the root of a lot of my writing. If you’re reading along, you’ll recognize that it’s at the heart of the fictional story I’m working on.

---

In the early days of my romance with Spotify, when I first got the sense that Spotify seemed to “know me,” I thought it was kind of cool. Wow, I’d think, as I hit several songs in a row that I really liked on the “Discover Weekly” playlist their algorithm created for me, they really get me. 

So I’d “like” some of these songs (in Spotify, you tap the heart button and it kind of pops and turns green and gets added to your “Liked Songs” list, a nice little dopamine UX sugarpop carefully designed by the developers. It’s the same process in the other music services, Apple Music, YouTube Music, Pandora, and … the other guys, whoever they are.) Pretty soon I’d create a long list of “Liked Songs” that I could play anytime, and every time I played it I could count on hearing things I liked.

Fast forward nine years though (I started using Spotify seriously in 2013), and it feels like I’m trapped in a musical hell, buried underneath a pile of songs and an algorithmically generated string of recommendations that once felt good but now feel like a cage. I used to think I was exercising my free will when I listened to music. Now, I’m not so sure. 

Every “Discover Weekly,” every “Release Radar,” every “Made for Tom Pendergast” playlist—which are basically all I see on my home page—seems to just add more and more of the same songs onto the pile. Remember the movie of Pink Floyd’s “The Wall,” when the bricks just kept getting higher and higher, and the poor guy is trapped? That’s me, trapped in my algorithmically generated pit of Americana.

So I determined, several months back, that I would dig my way out. I would assert my musical independence, and teach Spotify¹ that my musical tastes were way more eclectic than they “believed.” 

I considered two options: the first was to delete my account and start over—the “nuclear option”; the second was the more difficult task of teaching Spotify that I was something different than it believed. 

Delete and Start Over

Of course, the easiest thing to do if you don’t like who Spotify thinks you are, is to create an all new you: 

1. Delete your current Spotify account.

2. Start a new one (or better, start one with another music library).

3. Behave completely differently from a musical perspective.

Since Spotify and the other music services create your “musical identity” out of what you like, listen to repeatedly (or at least longer than thirty seconds), share with friends, add to playlists, etc., all you have to do is start again … but behave like a different person, the one you really want to be. (For a lot more detail on how Spotify creates your musical identity, see my Background section below.)

The Avett Brothers have a great song on the theme of starting over, by the way. Seems like a perfect time for a musical interlude.

Dig Out Slowly and Steadily

I knew I was stuck in a rut when I shared my “Your Top Songs 2019” playlist with my friend Kyle Burns and he said, “Tom, do you have a thing against women?” “No!” I answered. “Then why aren’t there any in your playlist?” I pointed out there were two Frazey Ford songs—but I had to admit he was right.

It was more than just the lack of women that was the problem though. When I listened closely, I had to admit that all my songs sounded the same, that all I was getting was a bunch of variations on “Come Pick Me Up” by Ryan Adams or “Should’ve Been In Love” by Wilco. It was time to teach Spotify to recognize a “new me.”

As I figured it, what I needed to do was to teach Spotify that I had changed. I needed to teach it to forget the “old me” and understand a “new me”—though how different the “old me” and the “new me” really are is up for debate. (Can people really change? That’s a whole ‘nother question!)

I’m going to explain the process I followed, not because I think my actions are inherently interesting but because I think the underlying question of how we relate to technology—and especially the ridiculously smart technologies that shape our culture and our identities—is vitally important to our lives. I for one like modern technology—but I do not want to be its pawn.

So the act of teaching Spotify to recognize a “new me” becomes a meditation on self-definition in the world of algorithmic recommendation engines and the behavioral prediction markets that shape modern surveillance capitalism.

But back to the music. Here’s how I’ve attacked the problem:

1. I rearranged everything that I had currently “liked” on Spotify. 

   1. First, I went through my entire “Liked Songs” list (it was 2,200 songs), moving all the songs I still wanted to listen to playlists (this is a hedge, however, not the purest approach I could have taken).

   2. Next I “un-liked” every song, so that Spotify no longer had a list of songs I had marked as “liked” (though of course they had my playlists and listening history).

   3. For several months, I totally ignored all the playlists I had created and ignored the Spotify-created playlists that they fed to me. I simply didn’t listen to what Spotify thought I liked. My goal was to teach Spotify that I had changed. Those songs and playlists belong to the “old me,” and while the “old me” isn’t dead, he’s not listening to the same old stuff.

   4. Whenever I did run across a song that fit the “old me,” I tried to skip past it; if I thought it appealed to the new me, I kept it playing. 

2. I listened to and “liked” new and different stuff, and let me tell you, this is the fun part. 

   1. I listened to music that’s way out of my existing rut. (That meant I had to stay away from the “Home” page, which is filled with everything Spotify thinks it already knows about me.) 

      1. I used search in Spotify to find new stuff. I started with “New Releases” and then looked for playlists pointing out stuff that’s not in my usual wheelhouse. I was trying stuff like “Latin New Releases” and “Jazz 2022” and the new album from Kendrick Lamar.

      2. I also sought out suggested music playlists from journalists, musicians, basically anybody who is treading different ground than me. Lots of music critics are publishing their Spotify playlists these days.

      3. My son Conrad is pretty good about tipping me to music he likes.

   2. I listened differently, more actively. 

      1. I started to “like” different things. Basically, when I heard something that felt like it had a different logic or feel than I was accustomed to, I either liked it implicitly, by lingering on a song for longer than 30 seconds, or explicitly, by “liking” it. I was teaching Spotify that I have different musical tastes by having different musical tastes!

      2. I also had to consciously skip forward on the music that sounds like the “old me,” even if I liked it. Sometimes that was hard!

   3. I built a new “Liked Songs” list.

      1. My ultimate goal was to create a “Liked Songs” list that reflects the “new me.” I assessed this list to see whether I really was deviating from my old habits. In my ideal world (one where I could turn the dials on the algorithms exactly as I wished), this “Liked Songs” list would be the strongest driver of Spotify’s suggestions–but I can’t tell you if that’s true or not. Spotify is pretty cagey about spelling out exactly how its algorithms work and I’m pretty sure its interests are not identical to my own.

3. I’m giving it time.

   1. Spotify didn’t learn to know the “old me” right away, and it won’t recognize the “new me” overnight either.

   2. I recognize that I’m in control of how disciplined I am with teaching Spotify what I like, and every time I find myself liking the same old stuff, I know I run the risk of slipping right back.

Does it work? Basically, yes. I’d say my “Liked Songs” list right now reflects a bunch of stuff I would not have been listening to a year ago. Are there still songs in there that sound like the “old me”? You bet. But for every Eddie Berman there’s a Faye Webster, for every José Gonzalez there’s a Stavroz and a Lomelda. It definitely doesn’t feel like every other song was written by Ryan Adams or Wilco. Your results will vary—but I’d love to hear how it goes.

There are more important questions than what’s in my “Liked Songs” list, of course. Like, do I really have agency in a world controlled by algorithms? Is my belief that I can exercise free will in this world hopelessly naïve? I would like to believe that I am not simply a tool of the corporate surveillance machine, even as I participate in and enjoy some of what it offers. But maybe I’m fooling myself. What do you think?

Background

I read a lot about how Spotify chooses songs, but I’ve chosen to tell a “personal” story about how to modify your musical world. I’d suggest if you want more on the technical and political elements of Spotify, you start with these sources.

Made to Be Found: Spotify’s attempt to help music creators and distributors place their music on Spotify in ways that help it get found by potential fans. I think this is pretty worthwhile, though recognize that Spotify’s self-interest influences their guidance. https://found.byspotify.com/intro 

Dmitry Pastukhov, “Inside Spotify’s Recommender System: A Complete Guide to Spotify Recommendation Algorithms,” Music Tomorrow: Written to help music professionals understand how the Spotify system works, this source goes really deep into all the ways Spotify knows you. I like how clear they are about Spotify’s interest in keeping people on the system. https://www.music-tomorrow.com/blog/how-spotify-recommendation-system-works-a-complete-guide-2022 

Hucker Marius, “Uncovering How the Spotify Algorithm Works,” Towards Data Science: Marius starts with the basics of how Spotify gets music to you–but he goes nearly as deep as Pastukhov and is even more mathy. https://towardsdatascience.com/uncovering-how-the-spotify-algorithm-works-4d3c021ebc0 

Alex Ross, “Reasons to Abandon Spotify That Have Nothing to Do with Joe Rogan,” The New Yorker: Smart take on some of the injustices associated with Spotify. https://www.newyorker.com/culture/cultural-comment/imagine-a-world-without-spotify 

Damon Krukowski, “The Big Short of Streaming,” Dada Drummer Almanach: Krukowski argues that Spotify’s “entire model of financial success is based on misinformation: the misinformation that music is valueless.” He’s not happy about that. 

The Big Short of Streaming

 

1. I want to be clear that teaching Spotify about my tastes in no way implies that I think the company cares about what any individual listens to—all they want is for me to stay subscribed and keep bringing them that sweet monthly recurring revenue.