• 0 Posts
  • 5 Comments
Joined 1 year ago
Cake day: June 30th, 2025
  • BrainBow65@lemmy.worldtoTechnology@lemmy.world•Anthropic logged net loss of over $80 billion in first half of 2026English
    61·
    3 days ago

    Good question! Unfortunately, I haven’t found a good source myself. I’ve been trying to piece it together based on my theoretical/mathematical neuroscience background and self-study about transformer architectures. The problem is the point neurons in transformers don’t map to neurons in cortex. You have to instead look at the bigger functions that are being computed by each system.

    The best single source to learn more about theoretical neuroscience is Neuronal Dynamics: From Single Neurons to Networks and Models of cognition. But its super math heavy. If you want to easier to digest videos I recommend Artem Kirsanov on youtube. For transformer architecture 3 Blue 1 Brown has a great series of videos about it on YouTube as well.

  • BrainBow65@lemmy.worldtoTechnology@lemmy.world•Anthropic logged net loss of over $80 billion in first half of 2026English
    8·
    3 days ago

    There are positives and negatives to using cells for computing. There are very complex systems that are responsible for making sure cells grow in the correct architecture and are supported property via blood vessels and glial cells. There is also definitely something to be said about glial cells themselves having a role in computation (whole other topic I won’t get into) not just neurons.

    If were just looking at the input output relationship with neurons we can model it pretty well using neuromorphic hardware which is designed to respond like a neuron. There are still gaps but and there are scientists who believe it isn’t close enough.

    Our current hardware can model neurons closely as well but its less efficient at it. The current AI models use point neurons which takes a lot less computing power but I don’t think they’re the way to AGI. Personally I’m a fan of the adaptive exponential leaky integrate and fire model or the spike response model

  • BrainBow65@lemmy.worldtoTechnology@lemmy.world•Anthropic logged net loss of over $80 billion in first half of 2026English
    451·
    3 days ago

    I’m just going to throw in my 2 cents on the AGI issue since I have a better understanding than average. I’m currently a computational neuroscience PhD student. I don’t work in theoretical neuroscience currently but that’s my hope after I finish my doctorate.

    I don’t believe current transformer architectures are capable of AGI and all big AI companies just seem to be adding more parameters to established transformer architectures.

    Tap for spoiler

    From what I understand transformer architectures act similarly to primary sensory cortices, they model complex probability densities. These companies appear to be banking on the idea that adding more training data, having smaller networks working together with different expertise, and having the train of thought (phonological loop) modeled as part of the PDF is enough to reach AGI. I disagree.

    While all of those things are interesting solutions to parts of the problem I don’t think they address some of the bigger issues.

    1. in biological intelligence memory recall is built into the same system as information processing. Associations between things are learned by simultaneously recalling and processing
    2. decision making and higher order processing do not fit into a transformer architecture
    3. temporal encodings are only made, in the transformer system, via the attention mechanism which is much less efficient than a more biological mechanism.