iPhone maker wants to catch up to its rivals when it comes to AI.

Apple wants AI to run directly on its hardware instead of in the cloud::iPhone maker wants to catch up to its rivals when it comes to AI.

  • Diabolo96@lemmy.dbzer0.comEnglish
    141·
    1 year ago

    It’s already possible. A 4bit quant of phi 1.5 1.5B (as smart as a 7b model ) takes 1Gb of ram . Phi 2 2.6B (as smart as a 13b model ) was recently released and it would likely take 2GB of RAM with 4bit Quant (not tried yet) The research only license on these make people not want to port them to android and instead focus on weak 3B models or bigger models ( 7b+) which heavily limit any potential usability.

    1. Apple could mimic and improve the phi models training to make their own powerful but small model and then leverage the fact that they have full knowledge and control over the hardware architecture to maximize every drop of performance. Kinda like how the some people used their deep knowledge of the console architecture to make it do things that seems impossible.

    Or

    1. The Apple engineers will choose, either due to time constraints or laziness to simply use llama.cpp which will certainly implement this flash attention and then use an already available model that allow its use for commercial purposes like mistral, add some secret sauce optimizations based on the hardware and voilà.

    I bet on 2.

    • RotaryKeyboardEnglish
      1·
      1 year ago

      Apple could mimic and improve the phi models training to make their own powerful but small model and then leverage the fact that they have full knowledge and control over the hardware architecture to maximize every drop of performance. Kinda like how the some people used their deep knowledge of the console architecture to make it do things that seems impossible.

      This has been Apple’s strategy for the last, what, 30 years or so. Yes, they love to buy software that is already proved and integrate it into their ecosystem (like Siri), but they get a competitive advantage when they build something that their competition can’t just replicate. I’m betting on 1! (Actually, I’m just hoping for 1, because it would probably be a better user experience.)