• 🐝bownage [they/he]@beehaw.org
    link
    fedilink
    English
    arrow-up
    5
    ·
    1 year ago

    I was going to ask which are transformer based but it’s actually nearly all of them (non grey) 😳. I thought BERT was still pre transformers. Goes to show how quickly things have evolved in recent years.

    • manitcor@lemmy.intai.techOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      yeah, the idea of making it transformers all the way down was key, each node is a complete Turing machine.