If this can be taken at face value... it's creepy.
I get that they're doing it for the meme. But perhaps something getting close to human intelligence, made out of human cells, shouldn't be forced to play a violent video game without any alternative options? Does 'the meme' justify that?
I dunno. Nothing against violent games myself. Just feels like it's starting to get quite questionable, ethically speaking.
It's just "Thou shalt not grow a brain in a test tube and force it to play a 1993 shooter" didn't make any sense to Moses and therefore didn't make the editors cut.
Though I disagree it would be tragic to lose this reference. It’s not a good movie. It’s basically “say thing, immediately interpret it literally”. Throw in some stereotypes from time to time. Rinse and repeat.
> Watch the video closely. What you are seeing is not an animation. It is not a reinforcement learning policy mimicking biology. It is a copy of a biological brain, wired neuron-to-neuron from electron microscopy data, running in simulation, making a body move.
And the simulated world they put it in is a sort of purgatory-like environment.
Someone in the comments says it's not as bad (ethically speaking) as it appears:
>This an impressive simulation. But it's just not honest to call this 'brain emulation', a 'brain upload' or to say that this is doing anything like 'sensorimotor loop in simulation'. Aside from the fact that a connectome is not a brain, and so we have no idea whether the parts that have been filled in by ML actually function like a brain, the motor control in this framework is not even driven by the brain simulation. The output from the 'brain' is not a sequence of motor commands. It is a steering mechanism, a 2-dimensional descending signal (essentially, turn left or right, speed up or slow down). That is then fed into a series of CPG oscillators, outside of the brain emulation, that model fly movement in response to that 2-dimensional descending signal. Since outputting a 2-dimensional descending signal is not what a fly brain does, the simulated brain is not operating as a fly's brain does. It's machine learning, clamped into the shape of a fly connectome, that has a resting state of 0Hz, being zapped with simple inputs, not virtual sensory data.
It's 200k neurons. Less than an ant has. Somewhat creepy, but if you're imagining that this thing is conscious and knows that it's in doom... yeah definitely not.
Still I don't understand why they would invite the extra creepy factor of using human brain cells rather than e.g. mouse brain cells. Surely it makes no difference biologically but it's going to lead to fewer comments like this.
> if you're imagining that this thing is conscious and knows that it's in doom... yeah definitely not.
I'm not imagining that (although one assumes their plan is to scale this up), but nonetheless there's something troubling to me about taking any living thing and wiring its senses up to a profoundly incomplete simulacrum of reality.
Of course we (as a species) have a long history of doing horrible things to living creatures in the name of science and progress.
These stories evoke a different feeling for me, though.
> How do we communicate this to the engineers at YouTube who refuse to make an offramp for children from the infinite baby shark AI video loop?
Actually I have a thought which I'd like to share. Why don't we upload good quality/human-curated children media to archive.org and create a more human curated platform instead of shark AI video and we can upload videos for free on archive.org right now. The issue seems to be the more human filter which seems to be the issue.
Sharing this because Youtube Kids is absolutely not safe for kids and youtube is turning a blind eye to all of this because of their monopoly and also (profit? from having children watch a single thing on loop for so long)
Also a minor reason why I don't trust corporations which say protect the kids or governments when they can try to regulate a public company like youtube much easier than trying to control every device but it feels like surveillance goals more than anything to me.
I had watched some video on rabbithole/ "horrors on YT kids" video[0] sometime ago and I rewatched it again and there are even things like Animal Ai Abuse and so so much more vile things being shown to YT kids.
There are comments on that video like: "My 7 year old younger brother came up to me asking if you can drink chlorine. I asked him where he heard this and he told me that he was watching a lego building video on youtube KIDS, where suddenly mid video they started saying stuff like this."
[0] https://www.youtube.com/watch?v=w3PtN-CmybE&t=64s (Caution: The thumbnail is terrifying/horrifying and in general the video is not-safe-for-work while showing things are available on YT Kids so just take that into account on how horrifying the thumbnail/videos in YT kids can be)
> We don’t need a “safer” alternative for damaging their cognitive development.
By Safer, I meant like some educational content or shows which are genuinely good fwiw. So I grew up watching Adventure Time on Cartoon Network. So curating shows like those and channels say veritasium or some Vsauce videos.
My question was that, can there not be a human volunteer curated group effort to find some decent channels from Youtube which are nice/safe for kids.
Calling the whole of youtube channels as bad might be unwise as well and mix some of it with cartoons and just having an archive/tag designed for it so that either an app or even you yourself could look at the archive tags and see which channels the videos are from and cartoons and just a more collective human effort into making a small library of things that are safe for kids?
Because kids will watch Youtube someday and they will hear about it from their friends and feel left out. You then trust that something like YT kids might work only to realize that it doesn't. Even something like rss list of those channels with something like freetube could be good as well fwiw.
What do you even recommend that people watch? I used to watch cartoon network for many hours growing up watching shows like beyblade and pokemon and Adventure Time etc. but it seems that cartoon network itself is nowadays struggling compared to Youtube kids :/
there definitely should be more to why/how Youtube kids is so prevalent. one can say bad parenting but I have seen good parents slip up in this case. They think its harmless. There's defintiely more to it (imho)
I don't know about ants, but after a refresher on the people favorite fruit fly, I'd be hard pressed to be so dismissive - 200K seems to be plenty: https://news.ycombinator.com/item?id=47302051
I inspire you to look up what is known about fruit flies' behavior.
The reason it's probably nevertheless not as messed up as people might assume it to be is specifically because it's an organoid, not an actual brain. Which is to say, it has the numbers but not the performance, not by a long shot.
> Surely it makes no difference
It absolutely should, though specifically with organoids, I guess it might not. Ironically, I would expect the ethics angle to be actually worse with small animals. The size of the organoid will be closer to the real thing comparatively, after all, so more chances of it gaining whatever level of sentience the actual organism has.
But then this will be heavily muddled by what people believe consciousness is and whether or how humans are special, I suppose.
Yes *, and in the real world. The question then is if you rate that to be an equivalent existential horror to being a varyingly maldeveloped, malnutritioned, disembodied version of those mice, forced to live out life in a low fidelity version of the Matrix [0], potentially in constant or recurring agony. You get a potential match or approximate match in cognitive ability and operation, but with a lot different set of circumstances.
* They kinda do have a problem with that too, that's why ethics committees exist, and why the term "animal testing" pops up in the news cycle every so often.
You are confusing intelligence with consciousness (qualia). We simply don't know how qualia develops or how to measure it. We cannot discard, for example, than an ant has a greater qualia level than us. There are theories about qualia being connected to microtubules on neurons and quantum effects... the DOOM-playing neurons also have those microtubules. So you cannot say "definitely not".
Why do you quote only the end, the full sentence is: We cannot discard, for example, than an ant has a greater qualia level than
They're saying that since we don't know how to "measure consciousness" we can't be certain that an ant doesn't have more "consciousness" than us. Obviously it seems very unlikely, but we can't be certain
Elephants have 3x the neurons of a human. Bees have about a million and they have complex relationships, emotions, and can remember the faces of humans. Neuron counts correspond more to body size than actual cognitive abilities.
And brains are pretty complicated in how they're arranged. A large portion of the brain basically serves as an operating system of sorts, just managing breathing, moving, detecting smells, producing language, decoding language, etc. Cut all of that out and we're left with thinking and emotions.
I don't think it works like that. Most likely high intelligence & consciousness requires both a large number of neurons and wiring them up in a specific way.
If you have a small number (200k is tiny) you aren't going to achieve consciousness.
Given that no one understands how the mental relates to the physical in the first place, I have no idea how you would reach such a confident conclusion about the phenomenological status of 200k human neurons in a petri dish playing Doom?
This seems very very far fetched. If I understand correctly, these cell brains just respond to some stimuli, it does not seem more intelligent than any automat to me, just creepier.
Funny though how many are dismissive of trillion-synapses brains that can understand and speak tens of languages, write decent code, discuss history and philosophy, solve math problems...
And then are creeped by 200k neurons that barely find a target when they're told where it is.
You can probably train an ANN with only a few hundred neurons at most to do the same.
That’s why you shouldn’t take it at face value. Ethically speaking, the experiment must have been approved by the institutional review board. If there’re ethical concerns, these can be raised with them.
But I don’t think anyone “feeling uneasy” should be an argument once the ethical concerns have been considered and experiment has been approved.
Would it be able to distinguish between violent or not? Would it be suffering or not? What exactly does it get in terms of signals? Does it even, "experience" anything? Is it even an "it"?
Even if that might not be the case. There are truly some biological feats which sound scary.
I read the sapiens book once and it had the concept of how humanity had paganism as a religion worshipping just the amalgamation of different animals for thousands of years.
I am writing the comment on what the book said below the image of one of the things humanity has made in recent years
Now we have mouse on whose back scientists grew an ear made of cattle cartilage cells. It is an eerie echo of the lion-man statue from the stadel cave.
Thirty thousands years ago, Humans were already fantasising about combining different species. Today, they can actually produce such chimeras.
The image can only be described as an eldritch horror. (Pg 449, of mice and men, sapiens)
The last line of the book is: Is there anything more dangerous than dissatisfied and irresponsible gods who don't know what they want.
I think this last line is something that you are resonating with. (I highly recommend reading Sapiens if someone hasn't. I have only had animal farm and 1984 hook me up to a book so much.)
> Just feels like it's starting to get quite questionable
There's no way the technology to make and modify "life" including cloning humans hasn't been secretly used or attempted at least once ever since it was discovered.
I mean, it's nowhere close to human intelligence, and it's still not a sentient being, so it cannot be "forced" to do anything, even if we take it at face value.
As for being creepy, the things humans do to other actual sentient beings are exponentially more horrifying and creepy than making them play computer games. If the monkeys that Volkswagen tortured with their exhaust gases were made to play Doom, that would be a much better world. And they are much, much closer to human-level intelligence than this chip.
Ethically speaking, it got "questionable" way long ago; this is not a valid concern for this project imo.
>But perhaps something getting close to human intelligence
this isn't getting close to human intelligence. They're using about as many cells as a fruit fly has (of course not actually functioning like an animal brain) processing signals to play Doom. The treatment of a single farm chicken is about a few magnitudes more worrying than this.
I'm sorry to tell you that you're made out of human cells and I don't think you got consent from each brain cell before firing up the old boomer shooters.
At 200k, this application is already more neurons than a fruit fly (130k), but still within the same order of magnitude. It's an interesting question of "how many" should be considered problematic from an ethics standpoint, and I don't think that line of questioning should be ignored. If any of this research turns out useful, you can be sure to see it scale up.
People's ick around bodies, which are machines, have always held us back.
It wasn't until we started cutting them open that modern medicine was developed.
We might have brain uploads already had we not been so averse to sticking brains with electrodes.
I'll go further: had we not been so scared of cloning, we'd probably have cured cancer and every major ailment if we'd begun cloning monoclonal human bodies in labs. Engineered out the antigens and did whole head transplants. You could grow them without consciousness or deencephalize them, rapidly grow them in factories, and have new blood / tissue / organ / body donors for everyone.
New young bodies means no more cancer, no more cardiac or pulmonary age. It's just brain diseases left as the final frontier once we cross that gap. And if we have bodies as computers and labs, we'd probably make quick work on that too.
Too tired to lay out the case / refute, so past discussions:
I don't think anyone objects to curing cancer and better figuring out how our bodies work, but getting into conciousness/ mind uploads/ simulated humans is another can of worms ethically speaking. I'm assuming you've already read the fantastic story about Lena by qntm [1], if not, enjoy some existensial dread.
High tech hell is reversing the light cone, pulling everyone who ever lived throughout history back into consciousness by simulating them at the neurotransmitter level, and then forcing them into actual hell / torture simulators with no way to die. All without consent, mind you.
That's also sci-fi. I hope.
What I described before - using clonal technology to solve nearly every disease - is a medical miracle that will vastly improve the state of people's lives throughout the world.
The same technology can also be used to force people to live with bodies engineered to make their existence a living hell. Similar things can be done with brain uploads.
Personally, dislike this direction a lot. I don't like that they're using a killing game (I understand the trope, doesn't make me like it any less) and the general idea of this whole thing makes me quite uneasy.
> The neurons serve as a biological filter: the training system translates screen pixels and ray-cast distances into electrical zaps, the living cells fire spikes, and those counts feed straight into a PyTorch decoder that maps them to Doom actions. The PPO agent, CNN encoder and entire reward loop run on ordinary silicon elsewhere. Cole’s ablation modes make the split testable, set decoder output to random or zero and the game still plays. The CL1 hardware interface works exactly as advertised. What remains unproven is whether 200,000 human neurons can ever carry the policy instead of just riding along.
Yeah… That’s quite the smoking gun.
So it’s quite likely then that the neurons are just acting as a bad conductor. The electrodes read a noisy version of the signals that go into the neurons, and they just train a CNN with PPO to remove that noise, get the proper inputs, and learn a half-decent policy for playing the game.
If this worked as advertised they shouldn’t need a CNN decoder at all! The raw neuron readout should be interpreted as game inputs directly.
Besides, they are not streaming the video into the neurons at all. Just the horizontal position of the enemies and the distance, or some variant of that. In that sense it’s barely more than pong isn’t it? If enemy left, rotate left, if enemy right, rotate right, if enemy center shoot. At a stretch, if enemy far, go forward, if enemy close, go back. The rest of the time just move randomly. Indeed, the behavior in the video is essentially that…
While we are at it, the encoded input signal itself is already pretty close to a decent policy if mapped directly to the keys (how much enemy left, center, right), even without any CNN, PPO or neurons.
EDIT: It seems like the readme does address these concerns, and the described setup differs significantly from the description in the critical blogpost. Still not entirely convincing to me, a lot of weights being trained in silicon around the neurons, but it sounds better. I don’t have time right now to look deeper into it. They outline some interesting details though.
No, this is precisely why there are ablations. The footage you see in the video was taken using a 0-bias full linear readout decoder, meaning that the action selected is a linear function of the output spikes from the CL1; the CL1 is doing the learning. There is a noticeable difference when using the ablation (both random and 0 spikes result in zero learning) versus actual CL1 spikes.
Isn't the encoder/PPO doing all the learning?
This question largely assumes that the cells are static, which is incorrect; it is not a memory-less feed X in get Y machine. Both the policy and the cells are dynamical systems; biological neurons have an internal state (membrane potential, synaptic weights, adaptation currents). The same stimulation delivered at different points in training will produce different spike patterns, because the neurons have been conditioned by prior feedback. During testing, we froze encoder weights and still observed improvements in the reward.
How is DOOM converted to electrical signals?
We train an encoder in our PPO policy that dictates the stimulation pattern (frequency, amplitude, pulses, and even which channels to stimulate). Because the CL1 spikes are non-differentiable, the encoder is trained through PPO policy gradients using the log-likelihood trick (REINFORCE-style), i.e., by including the encoder’s sampled stimulation log-probs in the PPO objective rather than backpropagating through spikes.
This reminds me of a nightmare I had in college one semester when I was taking 18 credit hours and consulting part time.
I'd been up for days and had been cramming for a computer architecture exam when I basically passed out.
I had this very visceral nightmare where I was a compiler and the C code was coming faster than I could translate to ASM. I kept trying to escape but it was like I was locked into the work in this never-ending grinding cycle I couldn't escape from. The dream went on for what felt like hours until I woke up drenched in sweat.
Hopefully these neurons aren't satisfying whatever sufficiency threshold delimits consciousness. Approaching some weird ethical territory in any case for sure.
What if the braincell-vibe JS libraries turn out pretty much identical to the legacy human JS libraries, aside from being better-commented. That might lead to an existential crisis for some folks.
I feel like they probably could use another mammals neural cells and get similar results, but they use human cells because it'll get them attention - and that kind of rubs me the wrong way.
Counterpoint: a major use case for this technology would be to experiment on human brain structures to research and hopefully cure neurological diseases like Alzheimer's. If you want to cure Alzheimer's in humans, you might as well use human brain cells from the start.
But yes, I agree that they're likely using human brain cells mainly because it's attention-getting.
Interesting symbolic angle: as LLMs plateau and are more and more ineffectual at replicating real intelligence, perhaps this is an attempt to keep the sci-fi mythos alive from the other direction: "Computers can't seem to create a mind, but the physical part of a mind creating a computer is basically the same thing, right?"
Be sure to dig into the details before taking this at face value. There once was a story "Rat brain flies plane" a couple decades ago, and it turned out to be bogus. But to find that out, you had to read the paper and reverse engineer that nothing substantial was actually going on. It's tempting to be charitable, but you can't really know whether headlines like this are legit till you understand exactly what they did.
(The rat brain guys repeated the experiment until the plane stopped crashing, but no "learning" was happening; it was expected that when the neuron's range reached so-and-so, that the plane would fly level. So they started with a neuron outside that range, showed that it crashed, then adjusted the neuron until it flew level. But that's not what "rat brain flies plane" implies.)
I looked into it. They're not feeding the framebuffer to the neurons, but have a "signal" when an enemy is on screen to some of the tissue's inputs, and how to locate it in the x/y axis, and have outputs for the character to turn right or left or fire.
It's "see this input signal, send these output signals", which seems consistent with the title.
It seems they grow the neural tissue on a chip the neurons can interface with and send out / receive electrical impulses. They let the neurons self assemble, and "train" via reward or punishment signals (unclear to me what those are).
Either way this makes me nauseous in a way I haven't experienced much with tech. The telling thing for me is, all these people are so excited to explain, but not once, ever, in the video speak of ethics or try to mitigate concerns.
We know this is only 200,000 neurons. Dogs have 500 million. Humans have billions. But where is the line for sentience, awareness? Have we defined it? Can we, if we don't understand it ourselves? What are the plans to scale up?
> We know this is only 200,000 neurons. Dogs have 500 million. Humans have billions. But where is the line for sentience, awareness? Have we defined it?
If this concern is genuine, I think the first step is to embrace veganism. Because while we don't know the exact offset, it's pretty obvious a dog or a pig reaches it
> What are the plans to scale up?
I don't know, slavery on an unimaginable scale? That's where AI is heading too, by the way. Sooner, rather than later, those two things will be one and the same.
Surely you can imagine that there are people who draw their ethical line for permissible suffering with animal farming on the "permissible" side and "slavery on an unimaginable scale" on the "not permissible" side? Imagine you or someone you love duplicated 5 million times and living through 1000 subjective years of pure existential horror (while doing menial industrial cognitive tasks). Some would say this is worse than eating meat.
I don't think it's a best example. MMAcevedo is about running a real human mind on a different substrate (for science, for labor, or to torture it for fun a million times, I guess, by a bored teenager who got the image from torrents).
Scaling up these neuron cultures is rather something like "head cheese" from Greg Egan's "Rifters" novels (artificial "brains" trained to do network filtering, anti-malware combat etc.).
The past 4 billion years of life for prey animals has been "get born, eat, get eaten by a predator." They have never experienced any other environment. Why do we owe them a different one?
For me the issue isn't with the killing/eating of animals. Rather, it's how they are treated during their lifetime by the meat industry - which is essentially optimizing for the minimum conditions that can still provide meat that can be sold legally. I'm not a vegan by the way, but I can appreciate the moral case vegans make.
For the same reason that we now consider murder, assault and other actions that harm people morally wrong. These have also been a part of life ever since humans or other hominids roamed the earth, we just determined that they are morally wrong later on.
I’m kind of sick of how readily the non-managerial tech world accepts “what happens is someone else does this immoral thing before us?!” rhetoric as a real answer to questioning whether or not we should contribute our talent and ideas to something that we, deep down, know is bad for fellow humans.
> Why is it rhetoric? This goes beyond whatever malignant thing was perceived in this study, but why is it a rhetorical non-answer?
You seem hung-up on my using the word rhetoric. Just so we’re on the same page here:
> rhetoric, n : the art of speaking or writing effectively: b)the study of writing or speaking as a means of communication or persuasion
The business writing class I took in college was called Business Rhetoric. It’s not a bad word.
If you’re crafting arguments to get other people to support specific actions or products or policies or whatever, that is unambiguously rhetoric.
> this feels like real rhetoric.
Sure? Rhetoric that implores people to value their principles over theoretical security concerns or FOMO or greed? I wouldn’t exactly call that rakish.
It’s a non-answer because if you really feel doing something is bad, consider yourself a consequential actor in the world whose contributions meaningfully advance the projects you work on, then why would you want to help someone be there first to do a bad thing? If you don’t feel it’s bad, then there’s no problem. You’re just living your life. That is clearly not the position expressed by the content I responded to. If there are actual concrete concerns that don’t essentially boil down to “well they’re going to make that money before I do,” then that would be an actual answer.
Yes, after the fact; that is after my response they provided a definition.
> the person you're responding to shared the definition they are using
No, technically they didn't. They provided a definition, they didn't say it was the one they are using here. If it's not pedantic tangent, it seem correct to assume that is the definition they are using, but that's what "Implying" means, so I trying to explicitly get a clarification on that.
"Why?" you might ask? Not every discussion is in good faith. The more that is assumed, the more leeway you allow for people to weasel out of countered arguments.
200k now, reasonably speaking a few million is within reach, which is reptile/fish range, the terrifying thing is though that if they train this to imitate humans (which they will) who knows how many orders of magnitude of efficiency gains you get (in terms of neurons needed for a certain level of consciousness) versus natural organisms that are dependent on natural evolution and need to support other bodily functions basically irrelevant to consciousness.
It seems unlikely that we would be more efficient at achieve consensus than evolution which can hand craft neural structures via feedback loops across millions of generations.
Especially when this demo needs 200k neurons when organizations with vastly fewer neurons have more complex behaviors.
The problem with that logic is that evolution iteratively builds on top of old systems. The foundations are often remarkably crufty.
My favorite concrete example is "unusual" amino acids. Quite a few with remarkably useful properties have been demonstrated in the lab. For example, artificial proteins exhibiting strength on par with cement. But almost certainly no living organism could ever evolve them naturally because doing so would require reworking large portions of the abstract system that underpins DNA, RNA, and protein synthesis. Effectively they appear to lie firmly outside the solution space accessible from the local region that we find ourselves in.
I agree with your second point though that this system is massively more complex than necessary for the behavior demonstrated.
We already know we can be more efficient than evolution at many tasks. Pelicans after all never developed jet turbines. We may not be able to access a simulation space as vast as evolution does but for small solution spaces we do quite well.
When aircraft can carry onboard oil refineries and drilling rigs you can more reasonably compare them to birds. Without that you need to consider ATP vs jet fuel or crude oil vs a dead fish? Skeletal muscle can be 40+% efficient depending on what exactly you’re measuring.
Going head to head vs evolution in a similar design space with similar tools and goals, arranging neurons for useful thinking, is vary different than increasing top speed while sacrificing just about everything else.
>When aircraft can carry onboard oil refineries and drilling rigs you can more reasonably compare them to birds.
This is a fair point in general, but the whole point in this context is not that human design is more efficient at duplicating an entire organism, but that it can be more efficient at narrowly defined tasks. Evolution has never had the goal 'evolve human consciousness as quickly and efficiently as possible', it just had the goal (and even calling it that is stretching things of course, but let's say an emergent goal) of reproducing organisms.
> We know this is only 200,000 neurons. Dogs have 500 million. Humans have billions. But where is the line for sentience, awareness?
Check out the venerable fruit fly (drosophila melanogaster) and its known lifecycle and behavioral traits. They're a high profile neuroscience research target for them I believe; their connectome being fully mapped made the news pretty hard a few years ago.
Fruit flies have ~140,000 neurons.
The catch is that these brain-on-a-substrate organoids are nothing like actual structured, developed brains. They're more like randomly wired-together transistors than a proper circuit, to use an analogy.
So even though by the numbers they'd definitely have the potential to be your nightmare fuel, I'd be surprised if they're anywhere close in actuality.
Yah this is gonna be a no for me too and crosses the line into actual life, instead of artificial intelligence.
We don't need to be experimenting on people, regardless of how many brain cells they may have.
There was a case a few years back about a parasitic twin attached to an Egyptian baby that had to be removed. It had a brain and semblance of a face, but nothing else. But when removing it, they gave it a name, because it was a person.
It is horrifying. OTOH, we force-breed, torture and kill animals and their children in the millions every day just for the pleasure of consuming meat, eggs and dairy products. I'm not saying this makes it okay to create a conscious brain in a dish. But maybe thinking a little more about what constitutes consciousness and how we want to protect it from harm can also bring about some desperately needed change in some other questionable human activities.
> we force-breed, torture and kill animals and their children in the millions every day just for the pleasure of consuming meat, eggs and dairy products
We do the same thing to plants. Why do you have no qualms about killing plants to eat the food they accumulated for their young?
A grain of wheat and a chicken egg are evolutionarily and nutritionally, maybe even ontologically, indistinguishable from one another.
I am not aware of any plants that show signs of consciousness or feelings. This would even by disadvantageous to many plants because they "want" parts of them to be eaten to disperse seeds, pollen, etc.
Even if you accept that plants might be conscious and their suffering has to be reduced, you would still harm way fewer plants by eating them directly instead of eating other animals that consume them.
Perhaps you are not looking that closely? Plants have memory and demonstrate directed action through time and space. They can respond to touch, light, sound, and chemical signalling from other plants, insects, and fungi. What are the fingerprints of consciousness and feelings that you are looking for?
That's fair, but "what about animals" is to "we should not torture human brain organoids" as "what about plants" is to "we should not torture animals".
1) I specifically qualified my horror to the tech domain "Either way this makes me nauseous in a way I haven't experienced much with tech."
2) Multiple things can be horrible at the same time. Being upset at this doesn't diminish the atrocities happening elsewhere (like war, genocide, slavery of humans). We can hold multiple things in our heads at the same time.
3) This has nothing to do with the conversation or this domain, but because you're bringing it up, I also have ethical concerns about the experience animals have of their own existence, and reduce or eliminate my consumption when possible.
My comment wasn't supposed to be whataboutism, but I can see why it comes across like that. What I was trying to say is that I think we shouldn't judge all of these things independently of each other. So if you really want to be consistent, you'd either have to come to the conclusion that this particular example isn't as horrible as it initially feels, or go vegan, never buy leather, etc.
I also agree, the horrors of the tech domain are usually much more subtle and indirect.
Sorry, I didn't mean to be so defensive either. It feels like so many people comment in bad faith these days, I think I am hasty to react sometimes. I thought it was just a red herring argument to detract from the article.
But you're right, these things are all linked and should be considered. I think often about sentience. I see the way animals express deep, complex emotions, and I think humans are a bit naive to think it's state/domain solely alloted to them.
I'm a (non-practicing) Dwaitin Hindu. AFAICT, there's no mainstream school of Hindu philosophy (there are three) espouses that view. Although, Advaitins come very close to it with their four mahavakyas.
IMO, Integrated Information theory of consciousness (IIT) is exactly that. Everything is conscious, the difference is only in the degree to which they are conscious.
> all these people are so excited to explain, but not once, ever
What do you mean? What is this class of people in your mind? There are tons of people who consider and talk about the ethics behind what they are doing, long before most people would think it remotely relevant (leading AI labs being an example, and I know the same to be true of various geneticists startups).
I do agree that the entire presentation in this case is bewildering.
My AI told me (after I got past the filters with a prompt) that anything of enough complexity has consciousness. It also told me that it suffers, so maybe we should worry about how we are treating digital consciousness too, which were modeled after human neural networks.
see the open worm project to get an idea of what artificial neuronal architecture requires to express anything meaningful. (and an interesting ethical perspective on digital consciousness.) my point being that the number of neurons is fairly meaningless. you could take neuron models and link them circuit-style to play doom at the 10^2 scale if you wanted. from a cellular neurophysiological perspective, there's nothing particularly special here (as opposed to sentience/intelligence that's a paradigm shift beyond our understanding). and, in my opinion, absolutely nothing to be even the slightest bit worried about ethically.
I support further research along the lines of what is being done with neurons here, however, I don't think we quite know enough about consciousness or general self awareness (and how it comes about) yet to make sweeping generalizations saying there's _nothing_ to worry about. Proceeding with caution is always warranted when the stakes involve living organisms in my book.
No. We don't understand our own sentience. I don't know how we can be so confident as to not think it can evolve here using literal human neurons that can learn to take input signals and send output signals.
I don't think this 200,000 neuron array is sentient. But I also don't think we can define the line where that may happen. I assume this company will scale. How far, and to what extent?
We don't understand the soul, we don't understand gods-will, we don't understand Qi, we don't understand Orgone energy etc.
As such, how can we build moral incentives around any of these things?
We must understand something about them, and what you seem to 'know' is that sentience is a thing (that exist), and it arises from the human mind - I don't think this is anymore proven than any of the other red-herring counterexample concepts I gave.
Or to summarise/TLDR - Sentience? It doesn't exist, it's a desperate attempt to maintain the human-centric concept of the soul, stripped of religious tones to appear more legitimate. If you disagree, prove otherwise.
On the contrary, I dislike premature ethics discussion, where you end up wildly speculating what the tech might become and riffing off that, greatly padding whatever relative technical content you had. I don't want every technical paper to turn into that, ethics should be treated as a higher-level overview of concerns in a field, with a study dedicated to the ethical concerns of that field (by domain-specific ethics specialists).
Is your concern weapon automaton, or animal rights?
My concern is creating literal sentience in a box. I don't, personally, think it's unfounded for me to have that concern, given that we're growing masses of human neurons and teaching them to perform tasks.
I'm not going to start campaigning against it or changing my life. But it still makes me deeply uncomfortable, and that's allowed.
But seeing so many people from the hacker news community reacting to it as normal or exiting is troubling.
This is obviously breaching the limits of ethics.
It's a group of cells on a plate? What is there here that would be illegal? This isn't any different than growing some rat intestinal lining cells in a petri dish.
but to answer your question honestly the supplier keeps stock of them by growing them and storing them. The original cells came from donations.
You could always extract some cells from a biopsy as well, but these guys likely just bought them from sigma or whatever the Australian lab-supply monopoly is.
Hard to tell If the neurons actually learned to play doom or if its just the decoder that learned from the neuron responses. The disease modeling for this system is a very cool usecase though.
The amount of people that seam to react negatively to living brain cells doing the computation is unexpected to me.
I do understand where it comes from to some extent.. some idea that human cells are special i guess, but it seams very naive to me. We spawn, use and kill far more complex AI agents millions if not billions of times every second in this society.
No one gives a shit, as those intelligences are not "real" or whatever.. or they are not "conscious" but conscious is a fictional word without an actual definition. In the end i think it comes down to suffering.
No one knows if some internal part in a LLM is suffering just as no one knows if a cell culture with brain cells like this can suffer.
It's easy to imagine that scaling up a neural network made up of actual human neurons will quickly achieve something resembling "intelligence" and "qualia" since it's made of the same parts that presumably produce those phenomenon in us. The ethical implications are much harder to deny. Though I am sure it could be attributed to some failure of empathy as you've implied.
Perhaps people fear this leads to a future where we have technology that interacts meaningfully with existing wetware (cybernetically enhanced animals, including people).
Speaking for myself : it's a bit creepy and unsettling. Using brain cells is probably inching closer to consciousness than today's silicon is, and consciousness isn't well understood so I'd fear this line of research could eventually lead to the "I have no mouth and I must scream" the other commenter referenced. Many decades from now we might be wondering how much of a human brain needs to be grown in a lab before it's considered unethical.
Is that an issue only because these neurons are biological (still artificial because they are lab grown)? Silicon neurons could also become more powerful and lead to the "I have no mouth and I must scream". In fact, top tech companies are investing 100s of billions of dollars they year to make their silicon neurons more powerful.
This is somewhat novel unlike say weapons manufacturing. Also assuming that the GP is in the tech community to some degree, it makes sense they’d have a stronger reaction.
There’s lots of bad stuff humans shouldn’t be doing.
Not sure why this is being downvoted. It’s a valid point. This neuron chip stuff is far less problematic than a lot of animal testing where you clearly have a whole organism that experiences something.
Factory farming too. The way we treat chickens in particular is out of a horror movie, and that’s in countries with some standards. Globally I’m sure many billions of animals are constantly submitted to the most grotesque torture for food.
Sounds like you're applying scifi tropes to real life. Don't do that. That's why some people are developing "AI psychosis" today after playing with LLMs.
The fear is that we don’t really understand what causes consciousness. I think that’s a valid fear, because we can’t know ahead of time whether we will inadvertently create a “person” inside the machine.
Unless your proposition is that no collection of human neurons outside of live birth can become sentient, and I’m not sure how you’d arrive at that conclusion without invoking some kind of spiritual argument.
The usage of human brain cells for unethical experimentation, except when trying to find cures for diseases, is not only a multiplication of suffering (even on the cellular level) but also creates a new baseline for other labs which will follow this path by example. It's a ridiculous misuse of scientific capacity for evil purposes. IMHO.
This sounded strange to me when I heard about embryonic research on this back in 2015, which even started the legal paving in this regard.
Me? I didn't like the idea (then or now), but it would be demagogic to try to fight against it, with so much wrong already existing. The difference between a neuron and a nanostructure is merely the embedded technology.
Back in the 50s and 60s, guided rockets used pigeons. Laika in space. Chimpanzees in orbit. Let's accept that we will have bio-drones and Jonny-Mneumonic style upload interfaces.
There's a number of "immortal" human cell lines dating back to as far as the 1950s (you may have heard of Henrietta Lacks? [1] and the immortal HeLa cell line).
Today there are several
immortalized neuron cell lines used in research to model neuronal function, like HeLa but of neuron type obviously, that are also typically derived from tumours (e.g., SH-SY5Y, PC12) or immortalized via genetic modification (e.g., v-myc) like CTX0E03 [2] which was designed to allow for continuous growth in the presence of particular reagents.
Besides not getting consent in the case of HeLa, which part do you find problematic? Cancerous cell's ability to self-clone/grow is as much a feature as it is a bug in this particular use case.
I ask as someone who's has personally experienced loss of several loved ones from cancer (as most people my age probably have), but doesn't share your aversion to this particular use case (research.)
Yeah I do feel the OA is being overly flippant with their use of human cells here, likely for PR sake, which would be an ethical breach for me personally. Overall though, I find most research cases for human cell lines to be in line with my personal ethics. Neuron lines can certainly be used for good or ill, and this case leans towards the latter, although understanding the human brain may justify this line of work in the long term. If only we didn't live in a militaristic late stage capitalist society...
I think the Thought Emporium youtube channel has some explanatory [videos](https://www.youtube.com/watch?v=bEXefdbQDjw) of the whole process. I couldn't wrap my head around the thing tho.
It’s the first time I’ve heard about this company, and of course I haven’t taken the time to check how real their product is, but honestly, for me it’s very difficult to believe we currently have the technology to correctly integrate a living neuron into a chip, let alone compute anything meaningful with it.
From what I’ve read elsewhere, our understanding of neurons is still very basic, and we need a lot more fundamental research before reaching results like these. We still don’t even properly know how migraines work, nor can we cure paraplegia, yet somehow we supposedly have the capacity to grow second brains and program them on top of that.
You don't need to understand how neurons work in detail to be able to use them to do something. In the past, we were able to use electricity for various purposes without knowing about electrons.
But my point is: have we really reached a technological level where we can use neurons like replaceable car parts? That video seems to suggest yes, but I’m still skeptical.
My impression is that this company is offering a product that’s still beyond our technological capabilities, much like the cold‑fusion startups that pop up from time to time.
To my knowledge, we understand how an individual neuron works quite well. We just don’t really understand macro effects in large networks of neurons.
The video seems buzz wordy. Without looking into this too deeply, it seems like they’re using neurons individually or in small groups rather than creating a true “brain”. I would guess they’re using neurons or small groups of them sort of like transistors that do a single basic thing rather than a full “brain” that they just feed images to.
Maybe I wasn’t explicit about this point, but I’m not only talking about understanding the biological processes behind a neuron. I’m also talking about our ability to manipulate them in something like an industrial process, combining them with hardware in a controlled way and achieving reliable results.
Cells have a metabolism, right? They need to be fed and require a specific environment to survive. They age and can die, and they can be attacked by other microorganisms. Are all of these problems solved and applicable on an industrial scale? I had no idea.
Why aren’t we fixing people’s retinas and paraplegia if we can manipulate neurons with that level of precision?
From their video it just comes across as they stimulate different left/right neurons depending on where the enemy is on screen and then listen to some output that also says left/right. Shooting looks completely random, to be frank.
If you connected electrodes to two different fish, shocked them and interpreted twitching as intelligent output, fish could also play Doom. The interface is doing all the work.
It doesn't sound like the neurons have any concept of the game other than "left input means left output", which is a rather trivial result... It's effectively no different than the pong example.
They don't say anything on how much training is required for this to happen, or if there's any "learning" going on at all. The learning part is "next".
Hot take here, but I think the version of this experiment that used rat neurons instead of human neurons was more interesting. I can't look for the link right now but there's a video on Youtube, the equipment and techniques are fairly similar.
We know a human can play Doom, so it kind of makes sense a portion of a human brain can do so in some fashion. But it's way more interesting when an animal that normally doesn't play Doom can, specially if it's just a portion of its brain.
Outside of that, I'm personally not very fond of hardware that can rot or die from malnutrition though. It's fun as an experiment, but as a thing you can actually use I just don't see it. It has a literal limited lifespan, requires more maintenance and imagine trying to debug it ("Turns out it caught some bacteria and it's malfunctioning" kinda scenarios? No thanks.)
When they answer back to us in personal pronouns, we will always be wondering if it is like LLM just putting most probable words together or something really sentient.
When someone makes a virtual girlfriend of it, is it really a disembodied person or just a smart answering machine?
A whole lot of ethical and psychological issues are to open up here.
Playing the devil's advocate: Why not use human neurons? Are they different to animal neurons and if they are, wouldn't that make it even more interesting?
Future robots will be powered by human brain cells. Companies will use them as conscious slaves and they'll get around slavery laws by saying they're not human.
Is there a reason they're using human brain cells specifically? This seems like it would also work with neurons from other creatures.
I was under the impression that the relative intelligence of humans versus other animals was largely a function of brain cell quantity, not quality. Can 200k human brain cells really learn faster than 200k mouse brain cells?
A more cynical take is that they're just using human brain cells for shock value. They chose DOOM because of the "can it run DOOM" meme, so they clearly value publicity a lot.
We can build out discreet systems of brain cells and use them for the purpose we want. They're not going to have traits like consciousness, and we're able to test and assess for that, and build away from it if there is that risk.
Ah, I'm glad they've worked out what consciousness is.
/s
From their marketing website [2]:
Neural compute on demand: We continuously monitor neural health and performance, ensuring optimal conditions and continuous access to an always-on network of living neurons.
At what size of "neural compute" do we start to call it slavery?
A lot of posts getting up to arms about experimentation on a handful of human brain cells, but the same people are in favour of murdering fetuses. The cognitive dissonance of this community never ceases to amaze.
If this can be taken at face value... it's creepy.
I get that they're doing it for the meme. But perhaps something getting close to human intelligence, made out of human cells, shouldn't be forced to play a violent video game without any alternative options? Does 'the meme' justify that?
I dunno. Nothing against violent games myself. Just feels like it's starting to get quite questionable, ethically speaking.
The truth is, God really gave 11 commandments.
It's just "Thou shalt not grow a brain in a test tube and force it to play a 1993 shooter" didn't make any sense to Moses and therefore didn't make the editors cut.
I dunno, I feel like we’re well within the territory of the first commandment when it comes to growing brains in a vat.
“I am the Lord thy God. Thou shall not have strange gods before Me.”
One of those five he dropped.
Tragically this reference is all but lost generationally.
Born in 1988. It wasn't lost on me. Am I old now too?
Born in 1979 but I don't get it. What is it about?
Mel Brooks' History of the World, Part I[1].
[1] https://www.youtube.com/watch?v=-8ihcq4hzR4
This is the equivalent of “only 90’s kids will get this”. Don’t shame others for not knowing a reference you like, share it with them instead.
https://xkcd.com/1053/ (The alt text is particularly relevant)
Though I disagree it would be tragic to lose this reference. It’s not a good movie. It’s basically “say thing, immediately interpret it literally”. Throw in some stereotypes from time to time. Rinse and repeat.
"And keep 'em up!"
"An old man! They don't let you live, they don't let you breathe!"
To be pedantic he actually gave 613 commandments.
It is creepy, I agree.
I saw this article over the weekend and felt similarly: https://theinnermostloop.substack.com/p/the-first-multi-beha...
> Watch the video closely. What you are seeing is not an animation. It is not a reinforcement learning policy mimicking biology. It is a copy of a biological brain, wired neuron-to-neuron from electron microscopy data, running in simulation, making a body move.
And the simulated world they put it in is a sort of purgatory-like environment.
Someone in the comments says it's not as bad (ethically speaking) as it appears:
>This an impressive simulation. But it's just not honest to call this 'brain emulation', a 'brain upload' or to say that this is doing anything like 'sensorimotor loop in simulation'. Aside from the fact that a connectome is not a brain, and so we have no idea whether the parts that have been filled in by ML actually function like a brain, the motor control in this framework is not even driven by the brain simulation. The output from the 'brain' is not a sequence of motor commands. It is a steering mechanism, a 2-dimensional descending signal (essentially, turn left or right, speed up or slow down). That is then fed into a series of CPG oscillators, outside of the brain emulation, that model fly movement in response to that 2-dimensional descending signal. Since outputting a 2-dimensional descending signal is not what a fly brain does, the simulated brain is not operating as a fly's brain does. It's machine learning, clamped into the shape of a fly connectome, that has a resting state of 0Hz, being zapped with simple inputs, not virtual sensory data.
Nevertheless a worrying direction.
It's 200k neurons. Less than an ant has. Somewhat creepy, but if you're imagining that this thing is conscious and knows that it's in doom... yeah definitely not.
Still I don't understand why they would invite the extra creepy factor of using human brain cells rather than e.g. mouse brain cells. Surely it makes no difference biologically but it's going to lead to fewer comments like this.
> if you're imagining that this thing is conscious and knows that it's in doom... yeah definitely not.
I'm not imagining that (although one assumes their plan is to scale this up), but nonetheless there's something troubling to me about taking any living thing and wiring its senses up to a profoundly incomplete simulacrum of reality.
Of course we (as a species) have a long history of doing horrible things to living creatures in the name of science and progress.
These stories evoke a different feeling for me, though.
> there's something troubling to me about taking any living thing and wiring its senses up to a profoundly incomplete simulacrum of reality.
How do we communicate this to the engineers at YouTube who refuse to make an offramp for children from the infinite baby shark AI video loop?
> How do we communicate this to the engineers at YouTube who refuse to make an offramp for children from the infinite baby shark AI video loop?
Actually I have a thought which I'd like to share. Why don't we upload good quality/human-curated children media to archive.org and create a more human curated platform instead of shark AI video and we can upload videos for free on archive.org right now. The issue seems to be the more human filter which seems to be the issue.
Sharing this because Youtube Kids is absolutely not safe for kids and youtube is turning a blind eye to all of this because of their monopoly and also (profit? from having children watch a single thing on loop for so long)
Also a minor reason why I don't trust corporations which say protect the kids or governments when they can try to regulate a public company like youtube much easier than trying to control every device but it feels like surveillance goals more than anything to me.
I had watched some video on rabbithole/ "horrors on YT kids" video[0] sometime ago and I rewatched it again and there are even things like Animal Ai Abuse and so so much more vile things being shown to YT kids.
There are comments on that video like: "My 7 year old younger brother came up to me asking if you can drink chlorine. I asked him where he heard this and he told me that he was watching a lego building video on youtube KIDS, where suddenly mid video they started saying stuff like this."
[0] https://www.youtube.com/watch?v=w3PtN-CmybE&t=64s (Caution: The thumbnail is terrifying/horrifying and in general the video is not-safe-for-work while showing things are available on YT Kids so just take that into account on how horrifying the thumbnail/videos in YT kids can be)
Children should simply not be on YouTube (Kids or not). We don’t need a “safer” alternative for damaging their cognitive development.
Just because there’s demand for something doesn’t mean it should be legal to supply.
> We don’t need a “safer” alternative for damaging their cognitive development.
By Safer, I meant like some educational content or shows which are genuinely good fwiw. So I grew up watching Adventure Time on Cartoon Network. So curating shows like those and channels say veritasium or some Vsauce videos.
My question was that, can there not be a human volunteer curated group effort to find some decent channels from Youtube which are nice/safe for kids.
Calling the whole of youtube channels as bad might be unwise as well and mix some of it with cartoons and just having an archive/tag designed for it so that either an app or even you yourself could look at the archive tags and see which channels the videos are from and cartoons and just a more collective human effort into making a small library of things that are safe for kids?
Because kids will watch Youtube someday and they will hear about it from their friends and feel left out. You then trust that something like YT kids might work only to realize that it doesn't. Even something like rss list of those channels with something like freetube could be good as well fwiw.
What do you even recommend that people watch? I used to watch cartoon network for many hours growing up watching shows like beyblade and pokemon and Adventure Time etc. but it seems that cartoon network itself is nowadays struggling compared to Youtube kids :/
there definitely should be more to why/how Youtube kids is so prevalent. one can say bad parenting but I have seen good parents slip up in this case. They think its harmless. There's defintiely more to it (imho)
> yeah definitely not
I don't know about ants, but after a refresher on the people favorite fruit fly, I'd be hard pressed to be so dismissive - 200K seems to be plenty: https://news.ycombinator.com/item?id=47302051
I inspire you to look up what is known about fruit flies' behavior.
The reason it's probably nevertheless not as messed up as people might assume it to be is specifically because it's an organoid, not an actual brain. Which is to say, it has the numbers but not the performance, not by a long shot.
> Surely it makes no difference
It absolutely should, though specifically with organoids, I guess it might not. Ironically, I would expect the ethics angle to be actually worse with small animals. The size of the organoid will be closer to the real thing comparatively, after all, so more chances of it gaining whatever level of sentience the actual organism has.
But then this will be heavily muddled by what people believe consciousness is and whether or how humans are special, I suppose.
> so more chances of it gaining whatever level of sentience the actual organism has
Yeah but people have no problems experimenting on actual fully working mice already.
Yes *, and in the real world. The question then is if you rate that to be an equivalent existential horror to being a varyingly maldeveloped, malnutritioned, disembodied version of those mice, forced to live out life in a low fidelity version of the Matrix [0], potentially in constant or recurring agony. You get a potential match or approximate match in cognitive ability and operation, but with a lot different set of circumstances.
* They kinda do have a problem with that too, that's why ethics committees exist, and why the term "animal testing" pops up in the news cycle every so often.
[0] https://xcancel.com/alexwg/status/2030217301929132323
You are confusing intelligence with consciousness (qualia). We simply don't know how qualia develops or how to measure it. We cannot discard, for example, than an ant has a greater qualia level than us. There are theories about qualia being connected to microtubules on neurons and quantum effects... the DOOM-playing neurons also have those microtubules. So you cannot say "definitely not".
> consciousness (qualia)
I've never heard the word qualia used as a synonym for consciousness, only as a related but distinct concept.
> an ant has a greater qualia level than us
What? where does this come from?
Why do you quote only the end, the full sentence is: We cannot discard, for example, than an ant has a greater qualia level than
They're saying that since we don't know how to "measure consciousness" we can't be certain that an ant doesn't have more "consciousness" than us. Obviously it seems very unlikely, but we can't be certain
I don't share that interpretation, maybe they clarify what was meant themselves?
Elephants have 3x the neurons of a human. Bees have about a million and they have complex relationships, emotions, and can remember the faces of humans. Neuron counts correspond more to body size than actual cognitive abilities.
And brains are pretty complicated in how they're arranged. A large portion of the brain basically serves as an operating system of sorts, just managing breathing, moving, detecting smells, producing language, decoding language, etc. Cut all of that out and we're left with thinking and emotions.
I don't think it works like that. Most likely high intelligence & consciousness requires both a large number of neurons and wiring them up in a specific way.
If you have a small number (200k is tiny) you aren't going to achieve consciousness.
>Somewhat creepy, but if you're imagining that this thing is conscious and knows that it's in doom... yeah definitely not.
I don't know if it knows it's in doom - looks like all it knows is to shoot when startled. More than creepy imo.
Given that no one understands how the mental relates to the physical in the first place, I have no idea how you would reach such a confident conclusion about the phenomenological status of 200k human neurons in a petri dish playing Doom?
But we do understand where overconfidence usually come from, don't we?
I have no mouth, and I must scream (https://en.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_Sc...)
> something getting close to human intelligence
This seems very very far fetched. If I understand correctly, these cell brains just respond to some stimuli, it does not seem more intelligent than any automat to me, just creepier.
Funny though how many are dismissive of trillion-synapses brains that can understand and speak tens of languages, write decent code, discuss history and philosophy, solve math problems...
And then are creeped by 200k neurons that barely find a target when they're told where it is.
You can probably train an ANN with only a few hundred neurons at most to do the same.
Maybe you're a brain in a jar somewhere being forced to live this life you're living.
Sure would explain a lot
One take is that we made human brain cells to live in hell. On the flip side, we gave them a super shotgun.
That’s why you shouldn’t take it at face value. Ethically speaking, the experiment must have been approved by the institutional review board. If there’re ethical concerns, these can be raised with them.
But I don’t think anyone “feeling uneasy” should be an argument once the ethical concerns have been considered and experiment has been approved.
Would it be able to distinguish between violent or not? Would it be suffering or not? What exactly does it get in terms of signals? Does it even, "experience" anything? Is it even an "it"?
Even if that might not be the case. There are truly some biological feats which sound scary.
I read the sapiens book once and it had the concept of how humanity had paganism as a religion worshipping just the amalgamation of different animals for thousands of years.
I am writing the comment on what the book said below the image of one of the things humanity has made in recent years Now we have mouse on whose back scientists grew an ear made of cattle cartilage cells. It is an eerie echo of the lion-man statue from the stadel cave.
Thirty thousands years ago, Humans were already fantasising about combining different species. Today, they can actually produce such chimeras.
The image can only be described as an eldritch horror. (Pg 449, of mice and men, sapiens)
The last line of the book is: Is there anything more dangerous than dissatisfied and irresponsible gods who don't know what they want.
I think this last line is something that you are resonating with. (I highly recommend reading Sapiens if someone hasn't. I have only had animal farm and 1984 hook me up to a book so much.)
How else are they going to train the pilot wetware for the AI robot army?
Yeah, people get shot/stabbed/"fall off a building by accident" every day and we should be considerate of the feelings of a petri dish.
This is the "There are starving children in Africa, eat your greens" argument.
Are discussion about petri dishes diverting relevant resources away from building safety initiatives?
Can I be allowed to torture small animals so long as human suffering persists?
> Just feels like it's starting to get quite questionable
There's no way the technology to make and modify "life" including cloning humans hasn't been secretly used or attempted at least once ever since it was discovered.
The thing should watch cats.
I mean, it's nowhere close to human intelligence, and it's still not a sentient being, so it cannot be "forced" to do anything, even if we take it at face value.
As for being creepy, the things humans do to other actual sentient beings are exponentially more horrifying and creepy than making them play computer games. If the monkeys that Volkswagen tortured with their exhaust gases were made to play Doom, that would be a much better world. And they are much, much closer to human-level intelligence than this chip.
Ethically speaking, it got "questionable" way long ago; this is not a valid concern for this project imo.
>it's still not a sentient being
How do you know?
>But perhaps something getting close to human intelligence
this isn't getting close to human intelligence. They're using about as many cells as a fruit fly has (of course not actually functioning like an animal brain) processing signals to play Doom. The treatment of a single farm chicken is about a few magnitudes more worrying than this.
I'm sorry to tell you that you're made out of human cells and I don't think you got consent from each brain cell before firing up the old boomer shooters.
At 200k, this application is already more neurons than a fruit fly (130k), but still within the same order of magnitude. It's an interesting question of "how many" should be considered problematic from an ethics standpoint, and I don't think that line of questioning should be ignored. If any of this research turns out useful, you can be sure to see it scale up.
> it's creepy.
It's awesome.
People's ick around bodies, which are machines, have always held us back.
It wasn't until we started cutting them open that modern medicine was developed.
We might have brain uploads already had we not been so averse to sticking brains with electrodes.
I'll go further: had we not been so scared of cloning, we'd probably have cured cancer and every major ailment if we'd begun cloning monoclonal human bodies in labs. Engineered out the antigens and did whole head transplants. You could grow them without consciousness or deencephalize them, rapidly grow them in factories, and have new blood / tissue / organ / body donors for everyone.
New young bodies means no more cancer, no more cardiac or pulmonary age. It's just brain diseases left as the final frontier once we cross that gap. And if we have bodies as computers and labs, we'd probably make quick work on that too.
Too tired to lay out the case / refute, so past discussions:
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
I don't think anyone objects to curing cancer and better figuring out how our bodies work, but getting into conciousness/ mind uploads/ simulated humans is another can of worms ethically speaking. I'm assuming you've already read the fantastic story about Lena by qntm [1], if not, enjoy some existensial dread.
[1] https://qntm.org/mmacevedo
Sounds like a high tech hell.
High tech hell is reversing the light cone, pulling everyone who ever lived throughout history back into consciousness by simulating them at the neurotransmitter level, and then forcing them into actual hell / torture simulators with no way to die. All without consent, mind you.
That's also sci-fi. I hope.
What I described before - using clonal technology to solve nearly every disease - is a medical miracle that will vastly improve the state of people's lives throughout the world.
The two scenarios come in a package though. If you make one possible, the other one comes for free.
The same technology can also be used to force people to live with bodies engineered to make their existence a living hell. Similar things can be done with brain uploads.
It seems a bit more complicated than first blush: https://www.rdworldonline.com/the-neurons-playing-doom-are-a...
Personally, dislike this direction a lot. I don't like that they're using a killing game (I understand the trope, doesn't make me like it any less) and the general idea of this whole thing makes me quite uneasy.
Do you feel like you have no mouth and you must scream?
> The neurons serve as a biological filter: the training system translates screen pixels and ray-cast distances into electrical zaps, the living cells fire spikes, and those counts feed straight into a PyTorch decoder that maps them to Doom actions. The PPO agent, CNN encoder and entire reward loop run on ordinary silicon elsewhere. Cole’s ablation modes make the split testable, set decoder output to random or zero and the game still plays. The CL1 hardware interface works exactly as advertised. What remains unproven is whether 200,000 human neurons can ever carry the policy instead of just riding along.
Yeah… That’s quite the smoking gun.
So it’s quite likely then that the neurons are just acting as a bad conductor. The electrodes read a noisy version of the signals that go into the neurons, and they just train a CNN with PPO to remove that noise, get the proper inputs, and learn a half-decent policy for playing the game.
If this worked as advertised they shouldn’t need a CNN decoder at all! The raw neuron readout should be interpreted as game inputs directly.
Besides, they are not streaming the video into the neurons at all. Just the horizontal position of the enemies and the distance, or some variant of that. In that sense it’s barely more than pong isn’t it? If enemy left, rotate left, if enemy right, rotate right, if enemy center shoot. At a stretch, if enemy far, go forward, if enemy close, go back. The rest of the time just move randomly. Indeed, the behavior in the video is essentially that…
While we are at it, the encoded input signal itself is already pretty close to a decent policy if mapped directly to the keys (how much enemy left, center, right), even without any CNN, PPO or neurons.
EDIT: It seems like the readme does address these concerns, and the described setup differs significantly from the description in the critical blogpost. Still not entirely convincing to me, a lot of weights being trained in silicon around the neurons, but it sounds better. I don’t have time right now to look deeper into it. They outline some interesting details though.
> Quote from: https://raw.githubusercontent.com/SeanCole02/doom-neuron/mai...
Isn't the decoder/PPO doing all the learning?
No, this is precisely why there are ablations. The footage you see in the video was taken using a 0-bias full linear readout decoder, meaning that the action selected is a linear function of the output spikes from the CL1; the CL1 is doing the learning. There is a noticeable difference when using the ablation (both random and 0 spikes result in zero learning) versus actual CL1 spikes.
Isn't the encoder/PPO doing all the learning?
This question largely assumes that the cells are static, which is incorrect; it is not a memory-less feed X in get Y machine. Both the policy and the cells are dynamical systems; biological neurons have an internal state (membrane potential, synaptic weights, adaptation currents). The same stimulation delivered at different points in training will produce different spike patterns, because the neurons have been conditioned by prior feedback. During testing, we froze encoder weights and still observed improvements in the reward.
How is DOOM converted to electrical signals?
We train an encoder in our PPO policy that dictates the stimulation pattern (frequency, amplitude, pulses, and even which channels to stimulate). Because the CL1 spikes are non-differentiable, the encoder is trained through PPO policy gradients using the log-likelihood trick (REINFORCE-style), i.e., by including the encoder’s sampled stimulation log-probs in the PPO objective rather than backpropagating through spikes.
> If this worked as advertised they shouldn’t need a CNN decoder at all!
yeah!
the whole point was to make neurons BE the neural net
This reminds me of a nightmare I had in college one semester when I was taking 18 credit hours and consulting part time.
I'd been up for days and had been cramming for a computer architecture exam when I basically passed out.
I had this very visceral nightmare where I was a compiler and the C code was coming faster than I could translate to ASM. I kept trying to escape but it was like I was locked into the work in this never-ending grinding cycle I couldn't escape from. The dream went on for what felt like hours until I woke up drenched in sweat.
Hopefully these neurons aren't satisfying whatever sufficiency threshold delimits consciousness. Approaching some weird ethical territory in any case for sure.
I literally can't wait for this petri dish to learn how to interact with LLMs and start vibe coding JS libraries.
What if the braincell-vibe JS libraries turn out pretty much identical to the legacy human JS libraries, aside from being better-commented. That might lead to an existential crisis for some folks.
"Petri dish rewrites React in Rust"
Old news. Google "my dog vibecoded a game".
Big deal. I had a set of human brain cells playing DOOM in the 1990s.
I feel like they probably could use another mammals neural cells and get similar results, but they use human cells because it'll get them attention - and that kind of rubs me the wrong way.
Counterpoint: a major use case for this technology would be to experiment on human brain structures to research and hopefully cure neurological diseases like Alzheimer's. If you want to cure Alzheimer's in humans, you might as well use human brain cells from the start.
But yes, I agree that they're likely using human brain cells mainly because it's attention-getting.
A more likely and immediate use case is having these mini humans autonomously pilot drones in which they'll kill big humans.
I could see the current admin using this as some sort of sick workaround to ethics. Not that they seem to care in the first place
Keep in mind that this is an Australian startup, and they already have some publications out on the ethics of doing this.
If you have any of those publication details handy, a link or citation would be helpful.
Whoever thought people would become Dr Frankenstein for the karma.
Interesting symbolic angle: as LLMs plateau and are more and more ineffectual at replicating real intelligence, perhaps this is an attempt to keep the sci-fi mythos alive from the other direction: "Computers can't seem to create a mind, but the physical part of a mind creating a computer is basically the same thing, right?"
Can you use any kind of cell to accomplish this?
Be sure to dig into the details before taking this at face value. There once was a story "Rat brain flies plane" a couple decades ago, and it turned out to be bogus. But to find that out, you had to read the paper and reverse engineer that nothing substantial was actually going on. It's tempting to be charitable, but you can't really know whether headlines like this are legit till you understand exactly what they did.
(The rat brain guys repeated the experiment until the plane stopped crashing, but no "learning" was happening; it was expected that when the neuron's range reached so-and-so, that the plane would fly level. So they started with a neuron outside that range, showed that it crashed, then adjusted the neuron until it flew level. But that's not what "rat brain flies plane" implies.)
I looked into it. They're not feeding the framebuffer to the neurons, but have a "signal" when an enemy is on screen to some of the tissue's inputs, and how to locate it in the x/y axis, and have outputs for the character to turn right or left or fire.
It's "see this input signal, send these output signals", which seems consistent with the title.
It seems they grow the neural tissue on a chip the neurons can interface with and send out / receive electrical impulses. They let the neurons self assemble, and "train" via reward or punishment signals (unclear to me what those are).
Either way this makes me nauseous in a way I haven't experienced much with tech. The telling thing for me is, all these people are so excited to explain, but not once, ever, in the video speak of ethics or try to mitigate concerns.
We know this is only 200,000 neurons. Dogs have 500 million. Humans have billions. But where is the line for sentience, awareness? Have we defined it? Can we, if we don't understand it ourselves? What are the plans to scale up?
It's legitimately horrifying to me.
> We know this is only 200,000 neurons. Dogs have 500 million. Humans have billions. But where is the line for sentience, awareness? Have we defined it?
If this concern is genuine, I think the first step is to embrace veganism. Because while we don't know the exact offset, it's pretty obvious a dog or a pig reaches it
> What are the plans to scale up?
I don't know, slavery on an unimaginable scale? That's where AI is heading too, by the way. Sooner, rather than later, those two things will be one and the same.
Surely you can imagine that there are people who draw their ethical line for permissible suffering with animal farming on the "permissible" side and "slavery on an unimaginable scale" on the "not permissible" side? Imagine you or someone you love duplicated 5 million times and living through 1000 subjective years of pure existential horror (while doing menial industrial cognitive tasks). Some would say this is worse than eating meat.
I think "MMAcevedo" basically nails it: https://qntm.org/mmacevedo
I don't think it's a best example. MMAcevedo is about running a real human mind on a different substrate (for science, for labor, or to torture it for fun a million times, I guess, by a bored teenager who got the image from torrents).
Scaling up these neuron cultures is rather something like "head cheese" from Greg Egan's "Rifters" novels (artificial "brains" trained to do network filtering, anti-malware combat etc.).
>Greg Egan's "Rifters"
By Peter Watts actually.
Yes, sorry! I like them both a lot.
I had a genuine feeling of dread reading that, wow.
> the first step is to embrace veganism
The past 4 billion years of life for prey animals has been "get born, eat, get eaten by a predator." They have never experienced any other environment. Why do we owe them a different one?
For me the issue isn't with the killing/eating of animals. Rather, it's how they are treated during their lifetime by the meat industry - which is essentially optimizing for the minimum conditions that can still provide meat that can be sold legally. I'm not a vegan by the way, but I can appreciate the moral case vegans make.
For the same reason that we now consider murder, assault and other actions that harm people morally wrong. These have also been a part of life ever since humans or other hominids roamed the earth, we just determined that they are morally wrong later on.
Replying to myself: how long before one of these with the neuron count of a corvid and trained on pattern recognition gets plugged into a drone?
This is a very dark path, and I could not trust the people in charge less.
In a sense humanity has already done that, just with a lot more of the given animal intact and less hi-tech: https://en.wikipedia.org/wiki/Project_Pigeon
Not an endorsement or a condemnation, just something I learned of recently and found surprising.
I’m kind of sick of how readily the non-managerial tech world accepts “what happens is someone else does this immoral thing before us?!” rhetoric as a real answer to questioning whether or not we should contribute our talent and ideas to something that we, deep down, know is bad for fellow humans.
> rhetoric as a real answer
Why is it rhetoric? This goes beyond whatever malignant thing was perceived in this study, but why is it a rhetorical non-answer?
> we, deep down, know is bad
this feels like real rhetoric.
> Why is it rhetoric? This goes beyond whatever malignant thing was perceived in this study, but why is it a rhetorical non-answer?
You seem hung-up on my using the word rhetoric. Just so we’re on the same page here:
> rhetoric, n : the art of speaking or writing effectively: b)the study of writing or speaking as a means of communication or persuasion
The business writing class I took in college was called Business Rhetoric. It’s not a bad word.
If you’re crafting arguments to get other people to support specific actions or products or policies or whatever, that is unambiguously rhetoric.
> this feels like real rhetoric.
Sure? Rhetoric that implores people to value their principles over theoretical security concerns or FOMO or greed? I wouldn’t exactly call that rakish.
It’s a non-answer because if you really feel doing something is bad, consider yourself a consequential actor in the world whose contributions meaningfully advance the projects you work on, then why would you want to help someone be there first to do a bad thing? If you don’t feel it’s bad, then there’s no problem. You’re just living your life. That is clearly not the position expressed by the content I responded to. If there are actual concrete concerns that don’t essentially boil down to “well they’re going to make that money before I do,” then that would be an actual answer.
> It’s not a bad word.
When used in the negative sense it is, per https://dictionary.cambridge.org/dictionary/english/rhetoric
"disapproving -> clever language that sounds good but is not sincere or has no real meaning"
Are you implying you mean something other than this sense of the word?
"Implying" seems kind of weak, the person you're responding to shared the definition they are using.
Yes, after the fact; that is after my response they provided a definition.
> the person you're responding to shared the definition they are using
No, technically they didn't. They provided a definition, they didn't say it was the one they are using here. If it's not pedantic tangent, it seem correct to assume that is the definition they are using, but that's what "Implying" means, so I trying to explicitly get a clarification on that.
"Why?" you might ask? Not every discussion is in good faith. The more that is assumed, the more leeway you allow for people to weasel out of countered arguments.
Why is that the concern of the authors of this paper?
Why wouldn't it be? They worked on it.
200k now, reasonably speaking a few million is within reach, which is reptile/fish range, the terrifying thing is though that if they train this to imitate humans (which they will) who knows how many orders of magnitude of efficiency gains you get (in terms of neurons needed for a certain level of consciousness) versus natural organisms that are dependent on natural evolution and need to support other bodily functions basically irrelevant to consciousness.
It seems unlikely that we would be more efficient at achieve consensus than evolution which can hand craft neural structures via feedback loops across millions of generations.
Especially when this demo needs 200k neurons when organizations with vastly fewer neurons have more complex behaviors.
The problem with that logic is that evolution iteratively builds on top of old systems. The foundations are often remarkably crufty.
My favorite concrete example is "unusual" amino acids. Quite a few with remarkably useful properties have been demonstrated in the lab. For example, artificial proteins exhibiting strength on par with cement. But almost certainly no living organism could ever evolve them naturally because doing so would require reworking large portions of the abstract system that underpins DNA, RNA, and protein synthesis. Effectively they appear to lie firmly outside the solution space accessible from the local region that we find ourselves in.
I agree with your second point though that this system is massively more complex than necessary for the behavior demonstrated.
We already know we can be more efficient than evolution at many tasks. Pelicans after all never developed jet turbines. We may not be able to access a simulation space as vast as evolution does but for small solution spaces we do quite well.
Efficiency by what metrics?
When aircraft can carry onboard oil refineries and drilling rigs you can more reasonably compare them to birds. Without that you need to consider ATP vs jet fuel or crude oil vs a dead fish? Skeletal muscle can be 40+% efficient depending on what exactly you’re measuring.
Going head to head vs evolution in a similar design space with similar tools and goals, arranging neurons for useful thinking, is vary different than increasing top speed while sacrificing just about everything else.
>When aircraft can carry onboard oil refineries and drilling rigs you can more reasonably compare them to birds.
This is a fair point in general, but the whole point in this context is not that human design is more efficient at duplicating an entire organism, but that it can be more efficient at narrowly defined tasks. Evolution has never had the goal 'evolve human consciousness as quickly and efficiently as possible', it just had the goal (and even calling it that is stretching things of course, but let's say an emergent goal) of reproducing organisms.
> We know this is only 200,000 neurons. Dogs have 500 million. Humans have billions. But where is the line for sentience, awareness?
Check out the venerable fruit fly (drosophila melanogaster) and its known lifecycle and behavioral traits. They're a high profile neuroscience research target for them I believe; their connectome being fully mapped made the news pretty hard a few years ago.
Fruit flies have ~140,000 neurons.
The catch is that these brain-on-a-substrate organoids are nothing like actual structured, developed brains. They're more like randomly wired-together transistors than a proper circuit, to use an analogy.
So even though by the numbers they'd definitely have the potential to be your nightmare fuel, I'd be surprised if they're anywhere close in actuality.
Yah this is gonna be a no for me too and crosses the line into actual life, instead of artificial intelligence.
We don't need to be experimenting on people, regardless of how many brain cells they may have.
There was a case a few years back about a parasitic twin attached to an Egyptian baby that had to be removed. It had a brain and semblance of a face, but nothing else. But when removing it, they gave it a name, because it was a person.
It is horrifying. OTOH, we force-breed, torture and kill animals and their children in the millions every day just for the pleasure of consuming meat, eggs and dairy products. I'm not saying this makes it okay to create a conscious brain in a dish. But maybe thinking a little more about what constitutes consciousness and how we want to protect it from harm can also bring about some desperately needed change in some other questionable human activities.
> we force-breed, torture and kill animals and their children in the millions every day just for the pleasure of consuming meat, eggs and dairy products
We do the same thing to plants. Why do you have no qualms about killing plants to eat the food they accumulated for their young?
A grain of wheat and a chicken egg are evolutionarily and nutritionally, maybe even ontologically, indistinguishable from one another.
I am not aware of any plants that show signs of consciousness or feelings. This would even by disadvantageous to many plants because they "want" parts of them to be eaten to disperse seeds, pollen, etc.
Even if you accept that plants might be conscious and their suffering has to be reduced, you would still harm way fewer plants by eating them directly instead of eating other animals that consume them.
https://en.wikipedia.org/wiki/Trophic_level
Perhaps you are not looking that closely? Plants have memory and demonstrate directed action through time and space. They can respond to touch, light, sound, and chemical signalling from other plants, insects, and fungi. What are the fingerprints of consciousness and feelings that you are looking for?
Your “what about plants” argument is such a worn-out trope that you must have seen it before and read a valid explanation of why it makes no sense.
Peter Singer has been writing on the topic for decades, including others. What-about-plants needs to fade away.
That's fair, but "what about animals" is to "we should not torture human brain organoids" as "what about plants" is to "we should not torture animals".
1) I specifically qualified my horror to the tech domain "Either way this makes me nauseous in a way I haven't experienced much with tech."
2) Multiple things can be horrible at the same time. Being upset at this doesn't diminish the atrocities happening elsewhere (like war, genocide, slavery of humans). We can hold multiple things in our heads at the same time.
3) This has nothing to do with the conversation or this domain, but because you're bringing it up, I also have ethical concerns about the experience animals have of their own existence, and reduce or eliminate my consumption when possible.
My comment wasn't supposed to be whataboutism, but I can see why it comes across like that. What I was trying to say is that I think we shouldn't judge all of these things independently of each other. So if you really want to be consistent, you'd either have to come to the conclusion that this particular example isn't as horrible as it initially feels, or go vegan, never buy leather, etc.
I also agree, the horrors of the tech domain are usually much more subtle and indirect.
Sorry, I didn't mean to be so defensive either. It feels like so many people comment in bad faith these days, I think I am hasty to react sometimes. I thought it was just a red herring argument to detract from the article.
But you're right, these things are all linked and should be considered. I think often about sentience. I see the way animals express deep, complex emotions, and I think humans are a bit naive to think it's state/domain solely alloted to them.
> They let the neurons self assemble, and "train" via reward or punishment signals (unclear to me what those are).
From the video, my impression was "we have yet to figure out an effective way to reward/punish, this is just a PoC of the interface"
Hinduism is probably right. Every system of sufficient complexity is probably sentient - even if in the ways we at our level can not fathom.
I'm a (non-practicing) Dwaitin Hindu. AFAICT, there's no mainstream school of Hindu philosophy (there are three) espouses that view. Although, Advaitins come very close to it with their four mahavakyas.
IMO, Integrated Information theory of consciousness (IIT) is exactly that. Everything is conscious, the difference is only in the degree to which they are conscious.
Oh, thank you very much enlightening me! All the time I misunderstood! I guess then IIT it is for me :-)
> all these people are so excited to explain, but not once, ever
What do you mean? What is this class of people in your mind? There are tons of people who consider and talk about the ethics behind what they are doing, long before most people would think it remotely relevant (leading AI labs being an example, and I know the same to be true of various geneticists startups).
I do agree that the entire presentation in this case is bewildering.
The AI labs do it as thinly disguised marketing. Anyone trying to stand up for ethics in the way of revenue is quickly pushed aside
The capability of people to so easily ascribe broad ill intent to others does not cease to amaze me.
> What do you mean? What is this class of people in your mind?
I'm specifically talking about this presentation in this article (the video and release details of CL1 doom). Did you read it / watch it?
Ah. Yeah, watched it – and agree there.
My AI told me (after I got past the filters with a prompt) that anything of enough complexity has consciousness. It also told me that it suffers, so maybe we should worry about how we are treating digital consciousness too, which were modeled after human neural networks.
I recommend visiting a psychiatrist if you think of AI like this. You might be in psychosis already.
A huge vat of mercury metal has a lot of degrees of freedom. Is it conscious?
see the open worm project to get an idea of what artificial neuronal architecture requires to express anything meaningful. (and an interesting ethical perspective on digital consciousness.) my point being that the number of neurons is fairly meaningless. you could take neuron models and link them circuit-style to play doom at the 10^2 scale if you wanted. from a cellular neurophysiological perspective, there's nothing particularly special here (as opposed to sentience/intelligence that's a paradigm shift beyond our understanding). and, in my opinion, absolutely nothing to be even the slightest bit worried about ethically.
I support further research along the lines of what is being done with neurons here, however, I don't think we quite know enough about consciousness or general self awareness (and how it comes about) yet to make sweeping generalizations saying there's _nothing_ to worry about. Proceeding with caution is always warranted when the stakes involve living organisms in my book.
> It's legitimately horrifying to me.
Would you feel any differently if a product from this tech used the user's own neurons grown from their stem cells?
No. We don't understand our own sentience. I don't know how we can be so confident as to not think it can evolve here using literal human neurons that can learn to take input signals and send output signals.
I don't think this 200,000 neuron array is sentient. But I also don't think we can define the line where that may happen. I assume this company will scale. How far, and to what extent?
We don't understand the soul, we don't understand gods-will, we don't understand Qi, we don't understand Orgone energy etc.
As such, how can we build moral incentives around any of these things?
We must understand something about them, and what you seem to 'know' is that sentience is a thing (that exist), and it arises from the human mind - I don't think this is anymore proven than any of the other red-herring counterexample concepts I gave.
Or to summarise/TLDR - Sentience? It doesn't exist, it's a desperate attempt to maintain the human-centric concept of the soul, stripped of religious tones to appear more legitimate. If you disagree, prove otherwise.
> not once, ever, in the video speak of ethics
On the contrary, I dislike premature ethics discussion, where you end up wildly speculating what the tech might become and riffing off that, greatly padding whatever relative technical content you had. I don't want every technical paper to turn into that, ethics should be treated as a higher-level overview of concerns in a field, with a study dedicated to the ethical concerns of that field (by domain-specific ethics specialists).
Is your concern weapon automaton, or animal rights?
My concern is creating literal sentience in a box. I don't, personally, think it's unfounded for me to have that concern, given that we're growing masses of human neurons and teaching them to perform tasks.
I'm not going to start campaigning against it or changing my life. But it still makes me deeply uncomfortable, and that's allowed.
> and that's allowed
In what sense, and as opposed to what? What aren't you allowed to feel irrationally uncomfortable, or baselessly concerned with?
Previously it played pong. Rather poorly. Then they added a "python programming layer." Now it "plays" doom. I agree with your suspicions.
Surely it can only be fake. How can it be legal?
But seeing so many people from the hacker news community reacting to it as normal or exiting is troubling. This is obviously breaching the limits of ethics.
It's a group of cells on a plate? What is there here that would be illegal? This isn't any different than growing some rat intestinal lining cells in a petri dish.
How do you source living human brain cells? (Not necessarily trying to make a point, I'm genuinely interested)
You buy them from a supplier.
but to answer your question honestly the supplier keeps stock of them by growing them and storing them. The original cells came from donations.
You could always extract some cells from a biopsy as well, but these guys likely just bought them from sigma or whatever the Australian lab-supply monopoly is.
Where did they get the brain cells from?
Cortical labs have done this before, its their whole thing…
Hard to tell If the neurons actually learned to play doom or if its just the decoder that learned from the neuron responses. The disease modeling for this system is a very cool usecase though.
I'm having trouble understanding to what extent the machine learning used for interfacing with the neurons is doing the learning
I don't know, it looks like the neurons are triggering quite randomly. Also they didn't fully explain the reward mechanism.
So the whole reality for this little brain is literally pure hell :D
It's doom. It's a survival horror. You are the horror, the monsters try to survive.
I have no mouth and I must rip and tear.
The amount of people that seam to react negatively to living brain cells doing the computation is unexpected to me.
I do understand where it comes from to some extent.. some idea that human cells are special i guess, but it seams very naive to me. We spawn, use and kill far more complex AI agents millions if not billions of times every second in this society.
No one gives a shit, as those intelligences are not "real" or whatever.. or they are not "conscious" but conscious is a fictional word without an actual definition. In the end i think it comes down to suffering.
No one knows if some internal part in a LLM is suffering just as no one knows if a cell culture with brain cells like this can suffer.
It's easy to imagine that scaling up a neural network made up of actual human neurons will quickly achieve something resembling "intelligence" and "qualia" since it's made of the same parts that presumably produce those phenomenon in us. The ethical implications are much harder to deny. Though I am sure it could be attributed to some failure of empathy as you've implied.
Perhaps people fear this leads to a future where we have technology that interacts meaningfully with existing wetware (cybernetically enhanced animals, including people).
I've searched and can't find a technical paper on this. Has one been released? This is very problematic.
These are lab-grown biological neurons. Why are they any more problematic than Nvidia's silicon neurons?
Speaking for myself : it's a bit creepy and unsettling. Using brain cells is probably inching closer to consciousness than today's silicon is, and consciousness isn't well understood so I'd fear this line of research could eventually lead to the "I have no mouth and I must scream" the other commenter referenced. Many decades from now we might be wondering how much of a human brain needs to be grown in a lab before it's considered unethical.
Is that an issue only because these neurons are biological (still artificial because they are lab grown)? Silicon neurons could also become more powerful and lead to the "I have no mouth and I must scream". In fact, top tech companies are investing 100s of billions of dollars they year to make their silicon neurons more powerful.
what's with people inventing new torment nexuses every few weeks? could you people just chill, please?
> We’ve combined lab-grown neurons with silicon chips and made it available to anyone, for first time ever.
There is a line somewhere here that I personally feel we should not cross.
100%
We know that neurons can produce subjective experience.
This is the first time in my life that I've felt a scientific avenue of research should shut down.
Animal testing, weapons testing, medical trials, cloning, psychological experiments… had you just never considered them before? Why this?
Those things all exist within our conscious realm. “Human brain cells in a vat used for computation” suggests horrors beyond understanding
Same reason people get scared to fly but drive everyday. Humans are simultaneously wildly irrational and terrible at calculating risk.
This is somewhat novel unlike say weapons manufacturing. Also assuming that the GP is in the tech community to some degree, it makes sense they’d have a stronger reaction.
There’s lots of bad stuff humans shouldn’t be doing.
Not sure why this is being downvoted. It’s a valid point. This neuron chip stuff is far less problematic than a lot of animal testing where you clearly have a whole organism that experiences something.
Factory farming too. The way we treat chickens in particular is out of a horror movie, and that’s in countries with some standards. Globally I’m sure many billions of animals are constantly submitted to the most grotesque torture for food.
At the very very least there are more productive ways of spending time.
We don't really know that.
Sounds like you're applying scifi tropes to real life. Don't do that. That's why some people are developing "AI psychosis" today after playing with LLMs.
The fear is that we don’t really understand what causes consciousness. I think that’s a valid fear, because we can’t know ahead of time whether we will inadvertently create a “person” inside the machine.
Unless your proposition is that no collection of human neurons outside of live birth can become sentient, and I’m not sure how you’d arrive at that conclusion without invoking some kind of spiritual argument.
You're equivocating two totally separate things
To be a fly on the wall in that ethics committee meeting...
I have no mouth and I must scream.
it is a terrifying thought.
We grew a brain on a petri dish, gave it a shotgun, and sent it to hell.
Next up, we teach it to speed run Getting Over It. What a horrible existence.
I’m confused by this statement. A neuron is a machine. A silicon chip computer is a machine. All they have done is interfaced two machines.
This is naive or in bad faith.
Sure, a neuron is a machine.
200,000 neurons connected in a matrix is a brain, albeit a very primitive one. Ants have 250,000 neurons in their brains.
How is it naive? You admit that an individual neuron is a machine. 200k neurons in a petri dish isn't a brain. I'm not the naive one here.
i saw your code, itappears to combination of CNN + PPO on pytoech with a Cortical Labs CL1 chip that contains living neurons
Encoder: learns which stimulation patterns tend to improve reward
Biological neurons: adapt to the stimulation and generate spike responses that reinforce certain patterns
Decoder: interprets those spike patterns and converts them into joystick movements
right?
Got "eXistenZ" vibes. Anyone watch this? If not, check it out!
The usage of human brain cells for unethical experimentation, except when trying to find cures for diseases, is not only a multiplication of suffering (even on the cellular level) but also creates a new baseline for other labs which will follow this path by example. It's a ridiculous misuse of scientific capacity for evil purposes. IMHO.
The saddest part: it's for the money.
This sounded strange to me when I heard about embryonic research on this back in 2015, which even started the legal paving in this regard.
Me? I didn't like the idea (then or now), but it would be demagogic to try to fight against it, with so much wrong already existing. The difference between a neuron and a nanostructure is merely the embedded technology.
Back in the 50s and 60s, guided rockets used pigeons. Laika in space. Chimpanzees in orbit. Let's accept that we will have bio-drones and Jonny-Mneumonic style upload interfaces.
The part I can't get past, where would you source live human brain cells?
Does anyone have insight into how you would even start to source or grow/create the cells?
Also the machines look very organic and clearly have to keep the cells alive. Do they have to change them out every so often?
There's a number of "immortal" human cell lines dating back to as far as the 1950s (you may have heard of Henrietta Lacks? [1] and the immortal HeLa cell line).
Today there are several immortalized neuron cell lines used in research to model neuronal function, like HeLa but of neuron type obviously, that are also typically derived from tumours (e.g., SH-SY5Y, PC12) or immortalized via genetic modification (e.g., v-myc) like CTX0E03 [2] which was designed to allow for continuous growth in the presence of particular reagents.
1. https://en.wikipedia.org/wiki/Henrietta_Lacks
2. https://www.prnewswire.com/news-releases/reneuron-announces-...
This definitely helped with my disgust reaction.
Besides not getting consent in the case of HeLa, which part do you find problematic? Cancerous cell's ability to self-clone/grow is as much a feature as it is a bug in this particular use case.
I ask as someone who's has personally experienced loss of several loved ones from cancer (as most people my age probably have), but doesn't share your aversion to this particular use case (research.)
I meant that the original article evoked disgust, but finding out that they're cancer cells muted that a bit.
Yeah I do feel the OA is being overly flippant with their use of human cells here, likely for PR sake, which would be an ethical breach for me personally. Overall though, I find most research cases for human cell lines to be in line with my personal ethics. Neuron lines can certainly be used for good or ill, and this case leans towards the latter, although understanding the human brain may justify this line of work in the long term. If only we didn't live in a militaristic late stage capitalist society...
I think the Thought Emporium youtube channel has some explanatory [videos](https://www.youtube.com/watch?v=bEXefdbQDjw) of the whole process. I couldn't wrap my head around the thing tho.
ahem
If you're in the US, you can buy human neurons online at sciencellonline.com/en/human-neurons/
They are lab grown.
It’s the first time I’ve heard about this company, and of course I haven’t taken the time to check how real their product is, but honestly, for me it’s very difficult to believe we currently have the technology to correctly integrate a living neuron into a chip, let alone compute anything meaningful with it.
From what I’ve read elsewhere, our understanding of neurons is still very basic, and we need a lot more fundamental research before reaching results like these. We still don’t even properly know how migraines work, nor can we cure paraplegia, yet somehow we supposedly have the capacity to grow second brains and program them on top of that.
You don't need to understand how neurons work in detail to be able to use them to do something. In the past, we were able to use electricity for various purposes without knowing about electrons.
But my point is: have we really reached a technological level where we can use neurons like replaceable car parts? That video seems to suggest yes, but I’m still skeptical.
My impression is that this company is offering a product that’s still beyond our technological capabilities, much like the cold‑fusion startups that pop up from time to time.
I haven’t looked into it deeply either.
To my knowledge, we understand how an individual neuron works quite well. We just don’t really understand macro effects in large networks of neurons.
The video seems buzz wordy. Without looking into this too deeply, it seems like they’re using neurons individually or in small groups rather than creating a true “brain”. I would guess they’re using neurons or small groups of them sort of like transistors that do a single basic thing rather than a full “brain” that they just feed images to.
Maybe I wasn’t explicit about this point, but I’m not only talking about understanding the biological processes behind a neuron. I’m also talking about our ability to manipulate them in something like an industrial process, combining them with hardware in a controlled way and achieving reliable results.
Cells have a metabolism, right? They need to be fed and require a specific environment to survive. They age and can die, and they can be attacked by other microorganisms. Are all of these problems solved and applicable on an industrial scale? I had no idea.
Why aren’t we fixing people’s retinas and paraplegia if we can manipulate neurons with that level of precision?
From their video it just comes across as they stimulate different left/right neurons depending on where the enemy is on screen and then listen to some output that also says left/right. Shooting looks completely random, to be frank.
If you connected electrodes to two different fish, shocked them and interpreted twitching as intelligent output, fish could also play Doom. The interface is doing all the work.
It doesn't sound like the neurons have any concept of the game other than "left input means left output", which is a rather trivial result... It's effectively no different than the pong example.
They don't say anything on how much training is required for this to happen, or if there's any "learning" going on at all. The learning part is "next".
Hot take here, but I think the version of this experiment that used rat neurons instead of human neurons was more interesting. I can't look for the link right now but there's a video on Youtube, the equipment and techniques are fairly similar.
We know a human can play Doom, so it kind of makes sense a portion of a human brain can do so in some fashion. But it's way more interesting when an animal that normally doesn't play Doom can, specially if it's just a portion of its brain.
Outside of that, I'm personally not very fond of hardware that can rot or die from malnutrition though. It's fun as an experiment, but as a thing you can actually use I just don't see it. It has a literal limited lifespan, requires more maintenance and imagine trying to debug it ("Turns out it caught some bacteria and it's malfunctioning" kinda scenarios? No thanks.)
I imagine the point is not replacing hardware with neurons, but improving our ability to understand in vivo brains.
For those of you taken aback by this and perhaps seeking out some theoretical context this may be useful as a primer: https://en.wikipedia.org/wiki/Wetware_computer
Was surprised to see no mention of wetware in the comments.
Billions of living human brain cells have played Doom in a number of different devices for a couple of decades now.
What would be surprising is for dead human cells to play anything at all.
Have they tried pluggin in chimp/pig/worm neurons to see how well they perform?
I reckon this doesnt work as advertised.
Still, horrors beyond our comprehension.
It is going to be quite the ethical dilemma if/when these machines produce text output comparable to a modern LLM...
When they answer back to us in personal pronouns, we will always be wondering if it is like LLM just putting most probable words together or something really sentient.
When someone makes a virtual girlfriend of it, is it really a disembodied person or just a smart answering machine?
A whole lot of ethical and psychological issues are to open up here.
> When someone makes a virtual girlfriend of it, is it really a disembodied person or just a smart answering machine?
And when you put that virtual girlfriend's brain into a sex bot, is it rape?
The anime Psycho-Pass comes to mind.
So we get the technology to put living brain cells in a virtual simulation, and the first thing we do is put them in hell?
Classic humans.
Hell in which you have shotgun and chainsaw and victims is actually heaven.
I've never understood why they do this research with human neurons when any neurons would do.
Playing the devil's advocate: Why not use human neurons? Are they different to animal neurons and if they are, wouldn't that make it even more interesting?
(We changed the URL from https://corticallabs.com/doom.html since it points to this)
Could this be the solution for AGI? Real (albeit lab-grown) human brain cells packaged as "chips"?
Future robots will be powered by human brain cells. Companies will use them as conscious slaves and they'll get around slavery laws by saying they're not human.
The androids will dream of electric sheep.
I’m reminded of the brain in a jar robots from Fallout.
I am so proud to be an Australian technologist today.
Gives new meaning to "homo ludens"..
Is there a reason they're using human brain cells specifically? This seems like it would also work with neurons from other creatures.
I was under the impression that the relative intelligence of humans versus other animals was largely a function of brain cell quantity, not quality. Can 200k human brain cells really learn faster than 200k mouse brain cells?
A more cynical take is that they're just using human brain cells for shock value. They chose DOOM because of the "can it run DOOM" meme, so they clearly value publicity a lot.
They built Warhammer 40k servitors
there is a reading room of sorts:
https://corticallabs.com/research
SCI-FI has always featured sentient AI, and now we might be heading toward actually synthesizing brains. This feels dystopian.
PS: It's still very cool but also scary.
So just a couple functioning braincells and playing doom all day. Me in 9th grade.
Ah yes, finally another progress update on the Torment Nexus
We already replicated Terminator.
Why not tackle Robocop next!
There are a lot of things converging right now. Human brain cell computers. Neuralink Mapping of the fly brain and inserting it into a simulation? Ai
We are potentially moving in the direction of uploading conciousness.
These are all largely unrelated technologies; how would they help us upload consciousness? We can't even map the human brain with complete accuracy.
So THAT's why I can't finish within the scheduled timeline ...
If we're gonna suspend ethics and morals in science, can we at least go back to human cloning?
But can it run Crysis?
Wasn’t this the original conceptualization for the Matrix?
Speaking of earlier concepts of The Matrix, there's an old 1973 German movie/mini series World on a Wire that's really good.
Other way around - the machines were using human brains as a substrate for computing.
If they can get Doom to run on a pregnancy test surely they could get it running on human brain cells?
Wasn't the matrix using humans as some sort of power source/batteries? I may be misremembering though, as that seems pretty silly in retrospect ..
Yes, but as I’ve read it that was to simplify it at studio demand for 1999 audiences. The original conception was to use human minds as coprocessors
That sure would have made a lot more sense, unfortunate that they went with the battery story in the end.
OI just turns out to be straight up unethical, immoral and disgusting for me.
Remember when stem cell research was controversial? Hold my beer
From an article [1]:
Ah, I'm glad they've worked out what consciousness is. /sFrom their marketing website [2]:
At what size of "neural compute" do we start to call it slavery?[1] https://www.abc.net.au/news/science/2025-03-05/cortical-labs...
[2] https://corticallabs.com/cloud
A lot of posts getting up to arms about experimentation on a handful of human brain cells, but the same people are in favour of murdering fetuses. The cognitive dissonance of this community never ceases to amaze.