5 comments

  • hbogert 30 minutes ago
    Shame they are not including old crap like my RX580 which is doing some impressive stuff with Qwen 9B Models.
  • pil0u 37 minutes ago
    I don't know what basis these numbers are estimated on, but my M2 Ultra 128GB runs almost twice as fast as what's reported for the M3 Ultra 512GB.

    I wished there was such a tool before picking my hardware, it's useful. But the estimates needs to be somewhat correct.

    • albinstman 15 minutes ago
      Thank you for the feedback. What model/quant did you test so I can look in it?
  • bozdemir 38 minutes ago
    Model list is a bit limited, and whole harness affects the speed, so the libs used in harness and their versions might be useful to show for each individual model setup.
    • albinstman 10 minutes ago
      I’ll look into adding harnesses. The benchmark sources usually don’t contain harness so the problem is limited data. What models would you like to see?
  • etiennelescot 11 minutes ago
    [dead]
  • PIYUSHMSONI98 1 hour ago
    [dead]