AI will remain a massively parallel numerics affair with enormous data sets and monstrous memory bandwidth and network crossection. And according energy consumption. Jevon’s paradox will eat any efficiency improvements.
Only if LLMs are the only option. A paradigm change is coming. It’s like what happened when European and Japanese performance cars started to take on American muscle cars or SpaceX (yeah I hate the Nazitard too) started recovering rockets and reusing them, or PCs started replacing mainframe workstations…
Look at the enormous processing resources of biological brains. Human brain is 2% of body mass but 20% of baseline metabolism – this is very expensive evolutionary. Neural hardware used for LLMs or just any scientific numerics accelerator is just a bad reinvention. Your argument reminds me of Minsky’s “5 MIPS is enough for AI”. Nope. You have to track a lot of state, its relationships and refresh it all very quickly. Computation is expensive.
I do hope that all the LLM companies have research teams that are investigating alternatives to LLMs as we know it today, rather than just how to make the existing LLMs more efficient/better.
The whole technology of how LLMs work seems flawed to the core, e.g hallucinations.
AI will remain a massively parallel numerics affair with enormous data sets and monstrous memory bandwidth and network crossection. And according energy consumption. Jevon’s paradox will eat any efficiency improvements.
Only if LLMs are the only option. A paradigm change is coming. It’s like what happened when European and Japanese performance cars started to take on American muscle cars or SpaceX (yeah I hate the Nazitard too) started recovering rockets and reusing them, or PCs started replacing mainframe workstations…
Look at the enormous processing resources of biological brains. Human brain is 2% of body mass but 20% of baseline metabolism – this is very expensive evolutionary. Neural hardware used for LLMs or just any scientific numerics accelerator is just a bad reinvention. Your argument reminds me of Minsky’s “5 MIPS is enough for AI”. Nope. You have to track a lot of state, its relationships and refresh it all very quickly. Computation is expensive.
I do hope that all the LLM companies have research teams that are investigating alternatives to LLMs as we know it today, rather than just how to make the existing LLMs more efficient/better.
The whole technology of how LLMs work seems flawed to the core, e.g hallucinations.