Backing up Spotify
annas-archive.li
We backed up Spotify (metadata and music files). It’s distributed in bulk torrents (~300TB). It’s the world’s first “preservation archive” for music which is fully open (meaning it can easily be mirrored by anyone with enough disk space), with 86 million music files, representing around 99.6% of listens.

We backed up Spotify (metadata and music files). It’s distributed in bulk torrents (~300TB), grouped by popularity.

This release includes the largest publicly available music metadata database with 256 million tracks and 186 million unique ISRCs.

It’s the world’s first “preservation archive” for music which is fully open (meaning it can easily be mirrored by anyone with enough disk space), with 86 million music files, representing around 99.6% of listens.

  • Zombie@feddit.ukEnglish
    67·
    10 hours ago

    After Meta scraped all their books they have the perfect defense now. All they have to say is “we’re training a music AI” and they’re apparently untouchable.

    • Signtist@bookwyr.meEnglish
      8·
      1 hour ago

      Well, they have to say “we’re training a music AI” while slipping several million dollars into the pockets of the right people. Rich people don’t win legal battles by actually proving what they did isn’t illegal, they do it by discreetly paying people to say they did.

    • mic_check_one_two@lemmy.dbzer0.comEnglish
      1·
      6 minutes ago

      Yes, and it hasn’t been easy to dig up until recently. There were a few ways to search the “hidden” metadata fields that Spotify uses internally. But it definitely hasn’t been easy or straightforward.

      Those hidden fields are how Spotify recommends similar artists. You have a few bands on repeat with specific instruments, chord progressions, and singer vocal range? Gee, maybe you’ll enjoy other bands that are similar to that…

    • infinitesunrise@slrpnk.netEnglish
      29·
      14 hours ago

      Both. Per the SQL schema printed in the article, table track_audio_features has both fields tempo and key along with many other technicals. Worth checking out, it’s near the bottom of the page.

      • driving_crooner@lemmy.eco.brEnglish
        7·
        12 hours ago

        Would love lmao. Just bought a second hand VDJ and I’m starting to experiment with mixxx, and I don’t know is the style I like (latincore and adjacents) or if the BPM detected of mixxx isn’t that good.

        • Rai@lemmy.dbzer0.comEnglish
          3·
          3 hours ago

          Good on you for starting that up! I wish you much success in your mixing and/or producing journey!

    • skisnow@lemmy.caEnglish
      36·
      16 hours ago

      It says,

      The data will be released in different stages on our Torrents page:

      [X] Metadata (Dec 2025)

      [ ] Music files (releasing in order of popularity)

      [ ] Additional file metadata (torrent paths and checksums)

      [ ] Album art

      [ ] .zstdpatch files (to reconstruct original files before we added embedded metadata)

    • mic_check_one_two@lemmy.dbzer0.comEnglish
      1·
      2 minutes ago

      Not yet, but that’s the end goal. The tricky part is that they’re only offering bulk downloads for now, which means downloading a single artist or album would be difficult/impossible. You’d need to download the entire compressed file of like 300GB of music, then extract the specific songs/artists/albums you wanted. The goal for now is preservation, meaning they want to make the bulk download as easy as possible, to make sure people can preserve it. Once they’ve got that in a pretty good spot, they may look into allowing more granular downloads.

    • AnarchistArtificer@slrpnk.netEnglish
      8·
      7 hours ago

      They’ve released torrents of the metadata, and they plan to release the music files, but they haven’t yet. They intend to start by offering the downloads as bulk torrents, but they’re open to considering implementing the ability to download single songs in the future.

      So in short, yes, but you can’t download them yet

    • notgold@aussie.zoneEnglish
      39·
      21 hours ago

      Would be amazing if it was. I would love to just have Spotify’s music on my nas

      • AstralPath@lemmy.caEnglish
        525·
        20 hours ago

        I’d wager 70% of what’s on Spotify is not worth preserving since its AI slop.

        • Hawk@lemmy.dbzer0.comEnglish
          131·
          13 hours ago

          Interestingly enough, with the data they provide, figuring out how much of it is AI slop wouldn’t be that hard I think

        • skisnow@lemmy.caEnglish
          172·
          15 hours ago

          Yeah as with most of the internet, it’s only worth downloading anything uploaded before 2023.

          So far, LLMs have done so much more harm than help.

        • nagaram@startrek.website
          English
          512·
          20 hours ago

          I’m not convinced AI slop can compete with the back log of organic slop personally.

          But yeah a fuckton is probably slop either way

          • bear@lemmy.blahaj.zoneEnglish
            2·
            1 hour ago

            AI slop is accelerating exponentially for the foreseeable future. It won’t take long for world data storage to be a limiting factor.

          • infinitesunrise@slrpnk.netEnglish
            10·
            14 hours ago

            A RAID6 of 24 * 20TB drives could contain that with both parity and hotswap, with room to spare. Let’s say $400 per refurb drive, $2500 rackmount SAS enclosure, $2000 SAS RAID card, $14,100 total. Assuming you already have the server and power and SAS cables.

            • wheezy@lemmy.mlEnglish
              3·
              6 hours ago

              You could budget this way down. I run 10+2 12TB with Unraid. No reason for a raid card if it’s for archive and personal use.

              • infinitesunrise@slrpnk.netEnglish
                1·
                4 hours ago

                Oh totally, could do SATA instead of SAS too. I used to build out servers and render farms for motion graphics studios that needed the ability for multiple people to be doing high-bandwidth operations on the same network drive, and the above was just kind of the default offering.

              • brognak@lemmy.dbzer0.comEnglish
                1·
                4 hours ago

                100% this. People who store easily replaceable media on RAID are just throwing away money (unless you have a need for faster read/write). If it’s your family photos, copy of your in progress thesis, or other irreplaceable piece of info/content go for it.

                I have like 40tb Unraid NAS and I get asked pretty much every time I talk to someone about it how I do backups. Easy, I backup my *arr stack databases and in case of a failure I restore them and let it pull down everything over time. Which I have done in the past when I wanted to upgrade quality, easier for me to scrub it all and start over than make upgrade profiles and such.

                Or that’s what I would have done, now I mostly use DebridService du jour and Stremio :-)

                • wheezy@lemmy.mlEnglish
                  1·
                  2 hours ago

                  Nice sounds similar to what I do. I bought Unraid because everyone always suggested it. No regrets. It really makes things simple and have only had minor problems (usually related to docker). I had a HDD fail and recovery was easy (besides the 18 hours of rebuild). But the drive is emulated during repair so no real downtime.

                  Family photos I use cloud storage for. I could probably get away with just doing local because I do hate that I’m sure they are being used to train AI. But I’d have to do off-site storage somewhere anyway. Parity drives don’t really do well in the “fire” scenario. Don’t want to have any risk of losing photos of my daughter growing up.

            • oyo@lemmy.zipEnglish
              32·
              6 hours ago

              This gif is going to completely lose its punch in a couple years.

    • Blackmist@feddit.ukEnglish
      8·
      11 hours ago

      Spotify has lossless now. Although if you’re listening on anything with Bluetooth then you probably won’t notice anyway.

    • EccTM@lemmy.mlEnglish
      58·
      18 hours ago

      They said this in the linked blog post:

      A while ago, we discovered a way to scrape Spotify at scale.

      Seems like reason enough to choose to scrape Spotify to me.

    • Vespair@lemmy.zipEnglish
      3·
      11 hours ago

      Spotify claims to offer lossless quality on much of their catalog; is this claim false or is there something more I’m missing here?