Analyzing 8 years of my music history with SQL
Ever heard an album or a song for the first time and loved it so much you played it on repeat for three days straight? I wanted to dig into my listening history to find out which songs I loved, got sick of, and forgot about for years.
Good thing I use Last.fm obsessively. Last.fm is a service that "scrobbles"—keeps track of—every song you listen to past the halfway mark. I've been using Last.fm since 2006 and have accumulated a seriously extensive scrobble log.
Last.fm provides users with an API for accessing their scrobble history. I figured that if I scraped my entire listening history into a SQL database, I could better analyze my listening history.
Scraping the Last.fm API with Python
I wrote some Python to claw my data out of the internals of Last.fm, one page at a time. It uses the Requests and Dataset packages. I used iPython Notebook to assemble a docs-and-code mashup of doom.
Check out my writeup on scraping Last.fm data here!
(If you have iPython Notebook, you can download the notebook file here and run it yourself.)
Whatchu know 'bout me?
Once my data was scraped into my DB, I ran some queries to search for songs I played obsessively after I discovered them, and I found several of my music-listening phases, such as:
- that period during which I listened exclusively to "Weird Al" Yankovic,
- the landmark date in 2006 when I listened to Daft Punk's Discovery for the first time, and
- that time I saw Wicked with my high school band and listened to the soundtrack for a week straight.
This is awesome. I'm a stats nerd and I love the idea of a quantified self. Thank you, 14-year-old Matt, for signing up for Last.fm 8 years ago.
There's plenty more to find out. Like, this has been my favorite song so far this week and this is my favorite newly-discovered album. And this is probably my favorite new artist by far.
OK, this is getting kinda creepy
But don't take my word for it—I've put my entire scrobble log online for you. Want to know on which day I fell asleep listening to Ellie Goulding in 2012? Want to dig up that evidence that I'm actually a closet Ke$ha fan? Download my SQLite database and learn what you will about 8 years of Matt's musical taste.
As always, email me at matt@mplewis.com or tweet me at @mplewis if you do something cool with this! I always love hearing when someone takes something I helped build and makes it even more interesting.
And if you do find out I like Ke$ha, please keep it to yourself maybe?