I’ll leave it to other comments to discuss the societal and moral implications of being able to do this (which, I agree, ick…). On a practical level:
We train an encoding model, a “digital twin”, that predicts how each visual region responds to any video. Now we can ask: which video would make a chosen region light up the most? NEvo searches for that video automatically, using the twin’s prediction as its reward.
I only scanned the paper, so maybe I missed it, but is there any confirmation that this ‘digital twin’ works? Like, do the generated videos actually cause the same patterns as in the ‘digital twin’ brain model in real humans in an MRI machine? My instinct is to be skeptical that it’s possible to reliably create a video -> brain activation prediction model.
All the signals that are missed by the (f)MRI are never "mirrored" in the digital twin even though on screen it will look like it. Bam, Experimenter Bias in the machine and I don't know how to phrase it but ... does this method/experiment leave any wiggle room for Falsifiability?
Even if the Hawthorne Effect does not apply to humans, it most certainly translates onto the brain and sensory (post)-processing-- live and remembered (vs in-memory, stored), real, virtual or imagined.
... I just realized how vastly different peripherals are when real, remembered, virtual or imagined sensory input is (post-)processed ...
It feels more like the machine will reproduce a superficial pattern. And at least some signal streams echoing in the brain on input from the digital twin won't emerge because the digital twin has only part of the data. And that's a premise for an unnoticeable rewiring of what once used to fire together, ... given a hell of a lot of exposure, of course, ... or not ... we'll see, I guess
This is the absolutely horrific next stage for social media platforms:
- They're already well able to surface the most addictive short video for a specific user out of millions of real videos.
- But these millions of real videos are just darts thrown into the space of "videos that could hook the user", in the end even the best-selected of them is not perfect.
- Now, behold! AI allows to generate the perfect video to surgically hit all the switches in the viewer's brain and turn it into a zombie hooked for days on end.
Let's hope our regulations hit these "social networks" hard enough so that never dare deploy this kind of technology.
Social media is not even the worst threat vector, in my opinion. It’s Anthropic/OpenAI/Google, whom people are giving an unprecedented level of access to their innermost thoughts.
We have not yet seen* the kind of large-scale, individually targeted psychological manipulation that cloud AI products can deliver. And I have no doubt the likes of Dario Amodei and Sam Altman will show us, if we give them enough time.
* I suppose the GPT-4o sycophancy/AI psychosis crisis was a preview, but that was just blunt “engagement” tuning.
The largest social media company in the world by several orders of magnitude is Meta, who are both using data collected from their social media platforms to train their AI models and improve ad targeting. Meta also offers AI chat bots on their platforms (FB Messenger and WhatsApp), to capture those innermost thoughts.
Don’t count out Zuckerberg, he’s very good at being a villain.
depends. I, for one, take control of what I consume on social media. Bluesky's design allows this. YouTube also, if you turn off recommendations, lose watch history, and vet your subscriptions
What makes you think they’d restrict this to logged in accounts? Not so long ago, Meta got caught building “ghost” social media profiles for people who didn’t have accounts on their platforms. They essentially constructed a “missing puzzle piece” based on people this person knew who did have profiles in publicly available information. Building a psychosocial profile on a user who has not set up an account yet using AI would be trivial.
Common misconception. We won't kill the planet even if we tried to. Wed make it too hard to sustain billions of humans, sure. But they are very different things
This is a good distinction because its a common idea but its based on a conflation: human civilizationhealth and planet health. Planet health is hard to even think about apart from humans but the planet in no way “needs” us.
Fortunately that particular extreme scenario doesn't seem likely. Radiance scales with the fourth power of temperature, and we're not filling the atmosphere with Venus levels of sulfuric acid. Our CO2 output will be self-limiting when it kills us and we stop outputting it.
there's not gonna be a regulation and even if there is, the damage would already be done. If you want to protect yourself, then best thing is to just delete the account and apps.
it interleaves the end note in the text at the point where the endnote is referenced (by number), reads the endnote, and then sounds a bell to indicate that we've returned to the main text.
Sometimes endnotes reference other endnotes and you have to wait until the point in the text, e.g. endnote 304 is referenced from earlier endnotes than it occurs in the text (rather late in the book) and you just have to wait until endnote 304 is read to find out what it is. that's a bit of a disadvantage, but infinite jest is the kind of book that you really need to read multiple times in order to get it anyways.
And yes, I agree. Im on my second read through now. The first time I skipped the first ~50 end notes. I thought the creator of infinite jest was a mystery, same with Joelles deformity (“was she really mangled or not— maybe we find out at the end!”). Oh, those two things are spelled out clearly in early end notes..
Any DFW fans here want to conjecture as to what “atemporal” jazz Pemulis listens to? I was thinking maybe some albert manglesdorf or perhaps ornette coleman..
I think parent is referring to the film in the novel, also named Infinite Jest. The film is so compelling that its viewers lose all interest in anything other than repeatedly viewing it, and thus eventually die.
you really have to catch the moment to do this, and there must be someone else who is supportive of this idea.
my mother recently got served a quite... disturbing video on Facebook. I won't get into details of the vid. what I can tell is that she was so dazed that she didn't even immediately understand me when I asked her, "delete Facebook and Instagram immediately or else they will think you like it and will serve you more." she kept replaying this video in her head. in a moment, I thought I wouldn't make her snap out.
for context, she is a damned Reels addict with war-induced traumas. (Ukrainian here.) she agreed to delete Facebook but decided to keep Instagram. we frequently pick bones about her sending stupid Reels and me hating them with passion. and every time I said "delete Instagram and your life will go up," she brought up scandals until she talked to a therapist.
EDIT: to quote a sibling comment, this is, indeed, like dealing with a drug addict.
I think you're giving this a bit too much credence. And I think you also think there's something special about sorting videos by how often they're viewed in a recent timeframe (which is all you need to recreate evil social media algorithm)
Carfentanyl potentcy is roughly 10,000 times morphine. The pursuit of optimized technology usually far outpaces regulation or any kind of control. As technologies are controlled, even the lessened availability of more potent types causes an enormous overall issue. Humans are obsessive-compulsive optimizers, for good and for ill.
Stop calling them “social” media. They have not been that since algorithmic feeds driving infinite scroll replaced updates from your friends. That was more than a decade ago.
There is nothing at all social about the latest generation like TikTok and YouTube Shorts.
If YouTube is social media, anything with a comments section is social media, even the old "guestbooks" of web 1.0 days were social media. It's a pointless term whoever uses it.
Youtube is very obviously social media. It is a platform that serves algorithmically sorted third party content based on user preferences and a network of accounts that a user follows, and it allows communication between users through comments. It has communities. It has memes. It is certainly addictive in all of the ways social media can be.
Don't forget it's not "the platforms" doing this, it's people like you and me, maybe even people reading this message. We, they, are building this, and don't care about anything as long the paycheck comes. This is the reality.
They have information barriers to keep the non-sociopaths from understanding the horror they're working on. Military forces do the same thing. Even the German train drivers didn't know what kind of facilities their trains were going to or why.
This is very similar to last week with that mind reading startup thing. Please read the paper before commenting.
This is a tool to help researchers in figuring out what different parts of the brain are actually for with less experimenter bias contamination of “well we think maybe it’s about this so let’s show it video of x to see”.
The essence runs on having someone sit in a scanner for a couple hours watching all sorts of things, and then feeding that to a model that will then build its own representation of said data and try different things on it until it’s found what makes a certain part sing in the model.
The purpose is a generalized understanding of brain function, more or less the same way we’ve been doing it all these years. Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
What is a "purpose"? Something people wish something would only be used for, right? How does it relate to, what influence does it have on what something will end up being used for?
> The synthesized clips line up with what each region is known to care about, faces for FFA, places for PPA, bodies for EBA, motion for MT, patterns for V1 / V3A, and lively social scenes for pSTS / aSTS
Explain the potential to exploit strong stimulation of specific visual regions for evil. "Oh, I very much detect a face/place/body/motion/pattern/human", says the subject. What are you going to do with that, startle them?
Apply the same technology several layers deeper. Stimulate the novelty or sexual attraction or general reward neurons. Stimulate the thirst center in a Pepsi ad.
>Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
It also helps companies like Moonbug Entertainment (Candle Media) understand how to build better Distractatrons.
It’s a small TV screen, placed a few feet from the larger one, that plays a continuous loop of banal, real-world scenes — a guy pouring a cup of coffee, someone getting a haircut — each lasting about 20 seconds. Whenever a youngster looks away from the Moonbug show to glimpse the Distractatron, a note is jotted down.
“It’s not mega-interesting, what’s on the Distractatron,” said Maurice Wheeler, who runs the research group. “But if they aren’t fully focused, they might go, ‘Oh, what’s that?’ and kind of drift over. We can see what they’re looking at and the exact moment when they got distracted.”
> p8: "A child watching television under normal conditions is subject to frequent interruptions and distractions. The TV must vie for his attention. In order to simulate this condition, we decided to program distractions into the laboratory situation... Slides could be used to fill the slide tray and they could be projected automatically, at regular intervals, onto a screen similar to that of the television set. The carousel projector allows the viewer to choose three exposure times. The 7.45 second interval proved most satisfactory with the preschool children.
And nowadays we think of Sesame Street as wholesome compared to today's content. They may have optimised attention but they were delivering positive-value content. That's not the case any more.
I can see it now.. The Distractatrons: a new chapter of protagonists in Transformers! The modern equivalent of evil in this day and age of ADHD and low attention span!
the same year as Baudrillard's Simulacra and Simulation, featuring computer generated characters, brainwashing television commercials, a light pulse gun that causes absence seizures, gun battles inside a plastic surgery clinic and an AR simulation environment, a sadistic computer, a physician who doubles as an action hero, and James Coburn giving a lecture explaining the enemy's evil plans in the opposite role that he played in the (excellent) The President's Analyst.
This time I was not so dazzled and saw it for as atrocious everyone else things it is. The minions of "Digital Matrix, Inc." manage several assassinations with the light-pulse L.O.O.K.E.R. gun but when they use real firearms they outdo Vader's stormtroopers by shooting each other. (Want to see the scene where somebody from E.Y. tells them to stick to the L.O.O.K.E.R. gun) The bad guys explain the penultimate secret to the protagonist early on but the ultimate secret is revealed in the L.O.O.K.E.R. lab which doesn't feel like a lab at all but rather a rather good room in a theme park experience where you're supposed to uncover the secret. (Contrast that to the lab Doug Trumbull outfitted in the Brainstorms movie a few years later which is packed with real surplus equipment... I've been to that lab!)
An anecdote, but some reason to think that overworking part of your brain is a bad idea:
I had an Aunt who had dementia. This is obviously a terrible outlook. But her and my Uncle seemed to be doing okay. My Uncle was an ultra-competent guy, highly stable, the person you could rely on; and had been his whole adult life. So it was shocking when he had a mental breakdown and became manic. There was probably something physical going on, but what was also going on was that he had wanted my aunt to be able to live normally for as long as possible, so had been covering everything - for a year or more he'd had to be alert 7 days a week in case my aunt tried to cook on the gas stove or something like that, at which she was no longer safe. So the risk-alert part of his brain had been constantly overworked.
I appreciate that this is scientific research, but there are definitely companies out there that will try to row-hammer everyone's brain if this sort of thing is not heavily controlled.
I'm not sure it's about overworking as much as stress - stress is a known potential trigger for manic episodes. Typically in people who already have some biological potential for experiencing it. And the stress of having your partner come down with dementia and suddenly becoming their full-time caregiver must be absolutely massive.
But re the underlying physical thing: You might look back and be able to spot some episodes of hypomania in your uncle's life. Perhaps just reduced sleep, or twinges of paranoia, or periods of on-again/off-again hyperproductivity, etc.
Hope that if this story is recent he's doing well. The recent generations of medications to prevent and treat mania are a huge improvement on the ones available in the past, so fingers crossed.
> I appreciate that this is scientific research, but there are definitely companies out there that will try to row-hammer everyone's brain if this sort of thing is not heavily controlled.
I will state plainly that I do not think you are asking this maliciously, and that instead you are asking from curiosity. From that perspective, though, I am a bit flabbergasted.
Are you truly curious as to how it might not be A Good Thing that those engaging in high-finance capitalistic money making programs would have access to forcefully engaging a human being's brain into activity, with or without their consent, to such an extent as to drive motivations and bias neuronal activation that leads to what they think is a choice?
Said another way: Are you really curious as to how allowing a company additional tools to bias human interactions may not be A Good Thing?
If you need an example as to how this is clearly not A Good Thing, and that at least a portion of the world population agrees with that characteristic, that there are currently companies that have tried to actively do this already, and attempts are being made at least to fine them for their behavior as an attempt to curb it. Whether or not it succeeds is a question of governance at the moment since the US is currently under plutocratic control.
However, I would really urge you to explore this topic and really engage with what's happening due to the plutocratic class' capabilities over technologic change and societally enforced or biased adoption, and how they, in many extreme cases that are continuing to become more extreme, are not A Good Thing.
> Are you truly curious as to how it might not be A Good Thing that those engaging in high-finance capitalistic money making programs would have access to forcefully engaging a human being's brain into activity, with or without their consent, to such an extent as to drive motivations and bias neuronal activation that leads to what they think is a choice?
I guess I just don't think this is the topic at hand. What you're proposing is mind control. That's not what's at issue. What's at issue is scientifically optimized superstimuli that can predictably activate certain brain regions. So when I read "rowhammer" I'm thinking about it in this context, the ability of a video to consistently activate a targeted section of your brain.
As you should know, and IIRC, cognition does not cleanly or consistently map geographically in the brain. To go from the ability to activate a subset of brain regions to influencing behavior is not a neglible step. The only real potential I imagine is in emotional activation, which is fairly well-localized.
So really what we're talking about is being able to evoke ideas and feelings with more consistency and precision than before. This doesn't seem new. And it can't be that intense either—I have the sense that your hypothetical mind control video would be extremely annoying or unpleasant before becoming influential. And for now we have the freedom to choose what we watch and turn the video off.
And it might be reasonable to say that you were referring to future possibilities, and that I'm focusing too much on what's currently available. But I am not just reacting to the first quoted clause of parent, but to their [vomits] policy recommendations. "This sort of thing", if not heavily controlled, etc. Maybe you see no value in this research, maybe there isn't any. Nonetheless the paranoid and the scifi-as-nonfiction readers will throw up the Bat signal for the idiots and moral panic orchestrators in power to wrap us all in comforting red tape for our own good.
I exaggerate. And I am talking about many more things than you were, as if the volume of text strengthens any rebuttal. All I'm saying is, this tech has little power over human behavior, in my opinion, and we should be cautious before bringing the hammer down before we know if we've got a bumblebee or a wasp.
This just strikes me as the false dichotomy that there are two option; complete freedom of choice, and complete mind control that you are physically incapable of resisting. That as long as you are not physically incapable of, say turning off a video, that there is no problem. Just because you can stop using crack doesn't mean it isn't dangerously addictive. Just because you can say no to someone holding a gun to your head does not mean it isn't incredibly coercive.
Also this seems to ignore that people can be influenced or manipulated without being consciously aware of it. Like the people that say that marketing doesn’t work on them, failing to see that every interaction with the products and services they buy and use are curated in some way to influence and manipulate them.
You just reminded me why I unsubscribed from ground news: you can't share a link without them adding their own paywall to it, even if it's a free article. Fuck that
I bet it wasn't overwork, it was the fight or flight response being constantly activated for your uncle when he's caring for your aunt who could do anything unpredictably.
Also: one of the V3A animations reminds me loosely of things I saw when I was a kid, at night, shortly before I slept (though my experience then was more circular).
How fried does someone's brain have to be to say "others on telegram", as if everyone is in the same chatrooms together? If they're on HN, they should at least know why that doesn't make sense, so I have to assume they're in a deteriorated mental state.
Friendly reminder to touch grass on a regular basis... Even virtual grass in a video game is a good pressure valve for excessive social media use.
Isn't there that one Harry Potter warning. I think it was the potion guy who said too much luck is dangerous. I guess that is somewhat of a parallel to this. Too much positive visual stimuli is dangerous or bad.
My understanding is that those who work with the mentally handicapped use bright lights and other stimuli to soothe and control them. It is also my understanding that the autistic are stimulated by vibrant colors (coughcoughMy Little Ponycoughcough).
Who is to say that the rest of us are not also vulnerable to such controlling stimuli?
There are undoubtedly 'adversarial images' that will induce certain effects in particular biological minds. It seems like this is neither good or bad inherently, just another tool.
I can imagine truly horrifying images coming out of this. Many people who have spent a lot of time online have come across unfortunate gore images that they may have seen for only a fraction of a second that they wish they could unsee. One can imagine the brain is able to see much worse.
June 9, 2032: ReMind, a new biotech startup, has opened to a recordbreaking IPO after a demo and a series of small-scale trials that stunned the world. The company is pioneering a new way to ingrain memories into the human brain - each ReMind screen is equipped with a special camera that can quickly recognize anyone who walks up to it, and ReMind's secret sauce algorithm is able to use past data of each user to generate imagery that will have the most profound impact on their brains. They claim that the algorithm is highly effective, being able to burn in imagery into the viewer's brains with an average accuracy of 83% per viewing. The CEO has promised to launch two services to start: ReMind Long-term, which focuses on creating imagery that simulates a 'traumatic' impression for creating long-term memories, which will be immensely useful for advertisers; as well as ReMind Short-term, for milder experiences that resemble a catchy song in a visual format for a quick boost in sales for a limited-time promotion. Advertising companies around the world have already placed tens of thousands of sales and deployment of ReMind-equipped screens is planned to begin immediately. Look forward to spotting one in your grocery store, in an elevator, on gas stations, on in-flight entertainment systems and in place of billboards on public spaces. The future is looking bright for ReMind!
This is already happening at scale by the social media feed algorithms. We don't need generated content to accomplish this. In a sea of user created content, plenty of it is already at peak activation.
Books are designed to be addictive. People said the same things about books as they say about screens nowadays. I think it's more important to understand the emotional hysteria people get themselves into
My favorite addiction epidemic was when apparently the "catch the ball on the string in the cup" took over parts of Europe to the point that people were ignoring their work.
I actually had a similar idea that I cam across in my own research in brain-to-video applications. You could theoretically train the model in real-time by using brain data as a reward signal, such that the model would find the image or video that most strongly produces the desired response (e.g. an increase in certain brainwave bands associated with pleasure, or alertness, or fear).
I can imagine this has some very undesirable usecases if used in the wrong way.
This does indeed seem comically evil. While surely this may provide somewhat interesting insights in how our brain processes things, this seems squarely past the "should" part of "you scientists were so obsessed with whether you could you failed to consider whether you should"
And I have yet to see a single paper like this where a researcher bails out and publicly says they refuse to work on such projects. Not one.
The most benign interpretation of this observation is that science is filled with spineless opportunists who don’t care who they hurt with what they create. A slightly less benign interpretation might be that many of these people are doing this deliberately, and getting off on the sense of power it gives them.
I was asked to train a neural network do detect how much pain a mouse was in- our partner company would be responsible for hurting and filming the mice. I refused and subsequently quit- this produced no paper and I don’t know if they got someone else to do it. I probably should have done something stronger.
When skip level bosses on my last job wanted to do boneheaded things in automotive design they usually had to keep asking different engineers until they got a yay.
When it is pushed from the top it is hard to stop at ground level.
In their defence, don't shoot the messenger. Just because they published it doesn't mean that others haven't already discovered it. Better to know its possible than be completely ignorant.
> Just because they published it doesn't mean that others haven't already discovered it.
That’s irrelevant for the moral evaluation. Everyone is responsible for their own actions. If you choose not to kill anyone it won’t stop others from killing. But the fact that this is so doesn’t give you a license to kill.
I'd say we're already well past that point. Short-form "content" already exists and is messing with people's brains, this is the same thing just taken a few steps further. By the time the tech companies start using it, it will already be too late and we'll be left discussing whether the next man-made nightmare they come up with is the point where the tech industry must be stopped.
The problem is the incentive structure. We could be doing good things with tech. The incentive is to use it to addict and destroy. Addiction and war is where all the money is, not education or health care or positive forms of entertainment and art. No money in those. Why?
I’m sure there was an early hominid version of this discourse. “Maybe bad to make sharp rock and sharp stick if this what we do with it…” “Mmm yes someday we make sharp rock big enough smash world.”
Advertisements actually run the world. Imagine a world without ads. The economy would grind to a halt. It is exploitation and manipulation, but that enables creation of large capital, which leads to great things. In the past it was pyramids and temples, and in modern times it is space exploration, scientific research and other things that require huge capital. Even the current advances in AI/LLMs are made possible by mass exploitation.
Without ads and exploitation of the masses, none of these would not be possible.
The pyramids were not built by slaves, and while we can waffle around about "all labor is exploitative to some extent", it doesn't take exploitation-maxxing to drive great achievements. Most great modern achievements are driven by a desire for self-actualization or recognition, not survival.
Slavery (or getting poorer people to do the work for you) is the central concept of civilization. It cannot be done without it. It's not capital or advertising that makes all this work, it's forcing the poor to work and pay tax that makes the world go round d
This reads like the mental hoops someone who works for or invests into Meta comes up with to resolve their cognitive dissonance.
You might as well have said that without advertising, we wouldn't have charities, freedom of thought, and literature: the argument is as strong and might make you feel even better about yourself.
Oh, turns out that you did say that advertising solved slavery in another comment. Carry on, mate.
Even if that is true, what you are saying is, essentially, that the ads-driven exploitative economy is a very expensive innovation engine.
It is then useful to ask: if innovation is what we want, do there exist engines that aren't quite as expensive?
If the market economy has more of a human face than outright slavery, there's no reason to believe that it's impossible to do better: a Copernican position would say that we'd be no more likely to be correct than an ancient Egyptian who claims it's impossible to do better than state slavery.
Please tell me how ads help science research or why should I care about space exploration or LLMs that threaten me and hundreds of thousands of other peoples jobs?
Can’t tell if this is satirical or serious. Those ancient wonders were enabled by slave labor, treating people as “capital” is the ultimate climax of capitalism.
I'm pretty sure by blindly following this path to commercial success in marketing; we're just gonna end up with content that is as implicitly adjacent to porn as possible without being explicit.
The title reads a lot like the lab logs you'd find in a horror game.
But for the paper itself, it seems they're using genetic optimization over predefined keywords. Wonder what would happen if they did gradient descent on the latent space directly. Is brain stimulation just not a good domain for GD?
Apart from ethically bad and evil use cases of this application, can we use it to massage the parts of brain like we do it to our bones and muscles with the help of physiotherapists?
reason I am asking it could be some relief to our brains after tedious working day, especially after heavy AI usage
Sure. The relief is actually free ; though once every 10 uses we'll spike your dopamine-related brain centers to 1,100% activation for a quick second while flashing a peptide sponsor ad, then back to normal.
You can of course, also not get the spike ads, by subscribing to Premium. Just have a quick look at this animated QR code, it will explain better than I can in writing.
Edit: there used to be an OSS app you could use where you actually pick what you want activated in your brain and when, but it's been banned after one of the commercial producers's investors convinced the government that this technology was too powerful, so only government-approved apps are allowed now, and an emergency vote passed the BrainControl law authorizing logging of all visual stimuli you look at, in order to protect your children.
Very interesting. We have an organic experiment converging to maximum stimulation in short form videos (which will become the majority of training data for future video gen models) Already approaching the capabilities of a “mood organ” from blade runner. Except usually most people don’t even make the choice to change their mood anymore.
https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_...
The sci-fi hand wringing is embarrassing. A huge chunk of the population is afflicted with mental health issues and the best psychitrists can seem to do is "Oh bro, wanna try some ketamine? Anti-psychotics?? Maybe some SSRIs, how about psychdelics?!" - this is a joke. Im all for more neuroscience research in every form because our contemporary understanding is embarassingly poor. This is a research tool for understanding what visual brain regions encode, not a mind-control device. Better models of brain function may eventually improve diagnosis and treatment of mental illness. Calling every neuroscience advance "Black Mirror" is not ethical insight. It saddens me the reaction so many commentors have here, it is just lazy technological illiteracy imo.
For a video to _literally_ maximally drive a target brain region there would be fatal consequences - e.g. the plot of David Foster Wallace's Infinite Jest (1996).
I wonder if the end-game of this field of research will be to run these simulations at scale using neuron-on-a-chip services such as [0] Cortical Cloud.
To flip the question on it's head, what exactly do you see as the "evil" potential? The most prominent comments to this effect are just scifi handwaving.
For one, it's bound to teach us something about our brains. And while I'm not usually an optimist, I actually find it more likely that we'll take active steps to stop the evil applications of this if research like this is out in the open.
Fuck this and whoever is driving this type of research. We’re increasingly living in a world designed by sociopaths. What do these people think this research will be used for?
I wish we had the Hippocratic oath for STEM, or at least that they would take ethics seriously rather than an afterthought against the god of Progress at all costs.
There are ethical use cases for this research. The first that comes to mind is mapping the brain before a surgery. Right now some surgeries are made with the patient concious to verify that relevant brain damages are not being done. Having a deeper knowledge of the patient's brain before the surgery may decrease risks further. The bad scenario (using targeted content to hook the user to a screen) already exists, it's not like it can get that worse (or at least so I hope)
Will be interesting to see how strong the controlling forces can be - enough to make you miss things in direct perception like in the book, or only softer effects further up the cognition layer stack
"Prime Intellect, I would like you to begin stimulating the neurons of the pleasure center of my brain, one at a time, and remember the ones I report to you as being favorable."
Really, I don't think you need to worry. A prevailing latent attitude in the comments is that the brain is somehow super malleable and subject to external control through some incantations. Like subliminal messaging, it will at most be moral panic that fades, and then people will act like they never believed in it.
My current theory is that these are similar to cigarettes. Nobody likes the first draft, it burns your lungs, your entire body wants to reject it. But the nicotine stimulates just the right receptors so that if you keep at it for just long enough, you'll be hooked and start disregarding the terrible taste, smell, tar in your lungs, and yellowing of your teeth.
All of this to say, if you subjected yourself to just enough TikTok scrolling on just the right topic, you might find yourself using it occasionally after that initial hump, then slightly more frequently, then daily.
You might still not "like" it, but the habit is what matters.
As a teenager, I tried to get addicted to cigarettes so I could stand by the cool kids and smoke with them. I started smoking 4-6 cigarettes a day but hated it so much, I couldn't continue after a week or two...
I hope my brain is also different. I also have never spent hours scrolling through short-form videos on Instagram, TikTok, Facebok, etc. I never ever walk outside with my phone in my hand, instead enjoying the view.
I do enjoy watching YouTube videos at home, on the living-room flatscreen, on a variety of topics, but I select them manually, one at a time, from the vast selection The Algorithm(TM) offers me, plus my own searches.
I am not sure if we should want to understand how our brain works. This is why I don't like libertarian free will people. We need to accept that we don't have free will. Otherwise we will study the brain thinking that we are above it. No. We are soon going to find out exactly how the brain works and I am not sure people understand what that means.
If you read the techbique it reads like something far less remarkable being PR’d to sound like a big deal.
The fact it’s bucketing by making images of lighting and facial expressions, the fact it doesn’t natively do the video it does an image then video generates from it.
The results look really bad and samey. Doubt this would work for the actual thing they’re pitching it for.
Customized propaganda made for the individual is the only thing I'm afraid of. Right now its easy to avoid the farm trolls that harves boomers minds on facebook, but given the costs go down and tech improves, we each of us have a trollfarm working specifically for us and that would make it extremely hard to not get brainwashed.
I’ll leave it to other comments to discuss the societal and moral implications of being able to do this (which, I agree, ick…). On a practical level:
We train an encoding model, a “digital twin”, that predicts how each visual region responds to any video. Now we can ask: which video would make a chosen region light up the most? NEvo searches for that video automatically, using the twin’s prediction as its reward.
I only scanned the paper, so maybe I missed it, but is there any confirmation that this ‘digital twin’ works? Like, do the generated videos actually cause the same patterns as in the ‘digital twin’ brain model in real humans in an MRI machine? My instinct is to be skeptical that it’s possible to reliably create a video -> brain activation prediction model.
... and shouldn't this backfire hard?
All the signals that are missed by the (f)MRI are never "mirrored" in the digital twin even though on screen it will look like it. Bam, Experimenter Bias in the machine and I don't know how to phrase it but ... does this method/experiment leave any wiggle room for Falsifiability?
Even if the Hawthorne Effect does not apply to humans, it most certainly translates onto the brain and sensory (post)-processing-- live and remembered (vs in-memory, stored), real, virtual or imagined.
... I just realized how vastly different peripherals are when real, remembered, virtual or imagined sensory input is (post-)processed ...
It feels more like the machine will reproduce a superficial pattern. And at least some signal streams echoing in the brain on input from the digital twin won't emerge because the digital twin has only part of the data. And that's a premise for an unnoticeable rewiring of what once used to fire together, ... given a hell of a lot of exposure, of course, ... or not ... we'll see, I guess
This is the absolutely horrific next stage for social media platforms:
- They're already well able to surface the most addictive short video for a specific user out of millions of real videos.
- But these millions of real videos are just darts thrown into the space of "videos that could hook the user", in the end even the best-selected of them is not perfect.
- Now, behold! AI allows to generate the perfect video to surgically hit all the switches in the viewer's brain and turn it into a zombie hooked for days on end.
Let's hope our regulations hit these "social networks" hard enough so that never dare deploy this kind of technology.
Social media is not even the worst threat vector, in my opinion. It’s Anthropic/OpenAI/Google, whom people are giving an unprecedented level of access to their innermost thoughts.
We have not yet seen* the kind of large-scale, individually targeted psychological manipulation that cloud AI products can deliver. And I have no doubt the likes of Dario Amodei and Sam Altman will show us, if we give them enough time.
* I suppose the GPT-4o sycophancy/AI psychosis crisis was a preview, but that was just blunt “engagement” tuning.
The largest social media company in the world by several orders of magnitude is Meta, who are both using data collected from their social media platforms to train their AI models and improve ad targeting. Meta also offers AI chat bots on their platforms (FB Messenger and WhatsApp), to capture those innermost thoughts.
Don’t count out Zuckerberg, he’s very good at being a villain.
What crisis and engagement tuning are you referring to?
I didn't realize that we'd start creating Westworld quite so quickly.
Social media is also AI now, so I don't really see the difference.
depends. I, for one, take control of what I consume on social media. Bluesky's design allows this. YouTube also, if you turn off recommendations, lose watch history, and vet your subscriptions
What makes you think they’d restrict this to logged in accounts? Not so long ago, Meta got caught building “ghost” social media profiles for people who didn’t have accounts on their platforms. They essentially constructed a “missing puzzle piece” based on people this person knew who did have profiles in publicly available information. Building a psychosocial profile on a user who has not set up an account yet using AI would be trivial.
We are doing to our brains and our children the same thing we are doing to pigs in cages, squezeeing every last cent until the planet burns down.
> until the planet burns down.
Common misconception. We won't kill the planet even if we tried to. Wed make it too hard to sustain billions of humans, sure. But they are very different things
Sure, and the planet survived the Permian-Triassic extinction event. Cold comfort to 70% of the terrestrial vertebrate species at the time.
The planet will still be here even if it burns down. After London burned down, there was still London. It was just burned down.
This is a good distinction because its a common idea but its based on a conflation: human civilizationhealth and planet health. Planet health is hard to even think about apart from humans but the planet in no way “needs” us.
You say that, but run away greenhouse gases can sterilize the entire planet, like Venus.
Fortunately that particular extreme scenario doesn't seem likely. Radiance scales with the fourth power of temperature, and we're not filling the atmosphere with Venus levels of sulfuric acid. Our CO2 output will be self-limiting when it kills us and we stop outputting it.
Sure, it will "only" look like another one of the Big Five extinction events.
We've already caused one of those, wiping ourselves out would actually end it.
We won't kill it, but we can try to burn it. Conflagration comes to mind.
Like a burned out brick building, a burned down planet is still there, but still also burned down. Of course it doesn't 'disappear'
Not even close chief.
Earth has been a rich, life sustaining planet in far hotter conditions then anything we are projected to take it to.
We can kill billions of ourselves, but short of ww3 earth will crack on just fine.
Social media emotional hot-takes dont match the numbers.
We are still screwed though.
there's not gonna be a regulation and even if there is, the damage would already be done. If you want to protect yourself, then best thing is to just delete the account and apps.
I am surprised that no comment has yet mentioned Infinite Jest (which I'm still yet to complete - it's been years since I started).
if you're interested in completing it, check out the 30th anniversary audiobook, it just came out a few months ago, very accessible
Curious how the audiobook deals with the end notes? They are crucial imo.
it interleaves the end note in the text at the point where the endnote is referenced (by number), reads the endnote, and then sounds a bell to indicate that we've returned to the main text.
Sometimes endnotes reference other endnotes and you have to wait until the point in the text, e.g. endnote 304 is referenced from earlier endnotes than it occurs in the text (rather late in the book) and you just have to wait until endnote 304 is read to find out what it is. that's a bit of a disadvantage, but infinite jest is the kind of book that you really need to read multiple times in order to get it anyways.
That seems reasonable! I’ll check that out.
And yes, I agree. Im on my second read through now. The first time I skipped the first ~50 end notes. I thought the creator of infinite jest was a mystery, same with Joelles deformity (“was she really mangled or not— maybe we find out at the end!”). Oh, those two things are spelled out clearly in early end notes..
Any DFW fans here want to conjecture as to what “atemporal” jazz Pemulis listens to? I was thinking maybe some albert manglesdorf or perhaps ornette coleman..
Care to elaborate for those that haven't started tackling this tome yet?
I think parent is referring to the film in the novel, also named Infinite Jest. The film is so compelling that its viewers lose all interest in anything other than repeatedly viewing it, and thus eventually die.
This occurred to rats who had levers wired directly to their reward centers.
> Let's hope our regulations hit these "social networks" hard enough so that never dare deploy this kind of technology.
You'd do as well to hope that people wake up and stop opening the damned sites
you really have to catch the moment to do this, and there must be someone else who is supportive of this idea.
my mother recently got served a quite... disturbing video on Facebook. I won't get into details of the vid. what I can tell is that she was so dazed that she didn't even immediately understand me when I asked her, "delete Facebook and Instagram immediately or else they will think you like it and will serve you more." she kept replaying this video in her head. in a moment, I thought I wouldn't make her snap out.
for context, she is a damned Reels addict with war-induced traumas. (Ukrainian here.) she agreed to delete Facebook but decided to keep Instagram. we frequently pick bones about her sending stupid Reels and me hating them with passion. and every time I said "delete Instagram and your life will go up," she brought up scandals until she talked to a therapist.
EDIT: to quote a sibling comment, this is, indeed, like dealing with a drug addict.
Ah, just like drug addicts put down their poppies...
Sounds like some people are trying to develop Snow Crash.
I think you're giving this a bit too much credence. And I think you also think there's something special about sorting videos by how often they're viewed in a recent timeframe (which is all you need to recreate evil social media algorithm)
If you're relying on government regulation to solve this, I guess the obvious bigger question would be "Quis custodiet ipsos custodes?
Carfentanyl potentcy is roughly 10,000 times morphine. The pursuit of optimized technology usually far outpaces regulation or any kind of control. As technologies are controlled, even the lessened availability of more potent types causes an enormous overall issue. Humans are obsessive-compulsive optimizers, for good and for ill.
Stop calling them “social” media. They have not been that since algorithmic feeds driving infinite scroll replaced updates from your friends. That was more than a decade ago.
There is nothing at all social about the latest generation like TikTok and YouTube Shorts.
If YouTube is social media, anything with a comments section is social media, even the old "guestbooks" of web 1.0 days were social media. It's a pointless term whoever uses it.
Youtube is very obviously social media. It is a platform that serves algorithmically sorted third party content based on user preferences and a network of accounts that a user follows, and it allows communication between users through comments. It has communities. It has memes. It is certainly addictive in all of the ways social media can be.
Algorithmic media
Now apply it to porn
Don't forget it's not "the platforms" doing this, it's people like you and me, maybe even people reading this message. We, they, are building this, and don't care about anything as long the paycheck comes. This is the reality.
Moloch wins again
https://medium.com/@darth.mark.23/the-moloch-dilemma-how-gam...
Speak for yourself, I would rather clean toilets for a living than work on these mind-destroying technologies
There's actual real people working on this bullshit. Horrifying doesn't even come close.
They have information barriers to keep the non-sociopaths from understanding the horror they're working on. Military forces do the same thing. Even the German train drivers didn't know what kind of facilities their trains were going to or why.
This is very similar to last week with that mind reading startup thing. Please read the paper before commenting.
This is a tool to help researchers in figuring out what different parts of the brain are actually for with less experimenter bias contamination of “well we think maybe it’s about this so let’s show it video of x to see”.
The essence runs on having someone sit in a scanner for a couple hours watching all sorts of things, and then feeding that to a model that will then build its own representation of said data and try different things on it until it’s found what makes a certain part sing in the model.
The purpose is a generalized understanding of brain function, more or less the same way we’ve been doing it all these years. Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
What is a "purpose"? Something people wish something would only be used for, right? How does it relate to, what influence does it have on what something will end up being used for?
> The synthesized clips line up with what each region is known to care about, faces for FFA, places for PPA, bodies for EBA, motion for MT, patterns for V1 / V3A, and lively social scenes for pSTS / aSTS
(STS seems to be https://en.wikipedia.org/wiki/Superior_temporal_sulcus in the temporal lobe, so I guess it's the "I sense a presence" region.)
Explain the potential to exploit strong stimulation of specific visual regions for evil. "Oh, I very much detect a face/place/body/motion/pattern/human", says the subject. What are you going to do with that, startle them?
Apply the same technology several layers deeper. Stimulate the novelty or sexual attraction or general reward neurons. Stimulate the thirst center in a Pepsi ad.
Anyone have a link or more info on the "mind reading startup" being referred to?
>Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
It also helps companies like Moonbug Entertainment (Candle Media) understand how to build better Distractatrons.
https://www.nytimes.com/2022/05/05/arts/television/cocomelon...https://www.newyorker.com/magazine/2024/06/17/cocomelon-chil...
What a world.
Sounds like they took a cue from Sesame Street:
> p8: "A child watching television under normal conditions is subject to frequent interruptions and distractions. The TV must vie for his attention. In order to simulate this condition, we decided to program distractions into the laboratory situation... Slides could be used to fill the slide tray and they could be projected automatically, at regular intervals, onto a screen similar to that of the television set. The carousel projector allows the viewer to choose three exposure times. The 7.45 second interval proved most satisfactory with the preschool children.
THE FIRST YEAR OF SESAME STREET: THE FORMATIVE RESEARCH https://files.eric.ed.gov/fulltext/ED047822.pdf
And nowadays we think of Sesame Street as wholesome compared to today's content. They may have optimised attention but they were delivering positive-value content. That's not the case any more.
I know a technology like that was used ~20 years ago for ADHD. EEG feedback, as soon as the kid looks away or zones out, the movie stops playing.
> ...the Distractatron
I can see it now.. The Distractatrons: a new chapter of protagonists in Transformers! The modern equivalent of evil in this day and age of ADHD and low attention span!
I recently rewatched the movie Looker which was vastly ahead of its time in 1981
https://en.wikipedia.org/wiki/Looker
the same year as Baudrillard's Simulacra and Simulation, featuring computer generated characters, brainwashing television commercials, a light pulse gun that causes absence seizures, gun battles inside a plastic surgery clinic and an AR simulation environment, a sadistic computer, a physician who doubles as an action hero, and James Coburn giving a lecture explaining the enemy's evil plans in the opposite role that he played in the (excellent) The President's Analyst.
This time I was not so dazzled and saw it for as atrocious everyone else things it is. The minions of "Digital Matrix, Inc." manage several assassinations with the light-pulse L.O.O.K.E.R. gun but when they use real firearms they outdo Vader's stormtroopers by shooting each other. (Want to see the scene where somebody from E.Y. tells them to stick to the L.O.O.K.E.R. gun) The bad guys explain the penultimate secret to the protagonist early on but the ultimate secret is revealed in the L.O.O.K.E.R. lab which doesn't feel like a lab at all but rather a rather good room in a theme park experience where you're supposed to uncover the secret. (Contrast that to the lab Doug Trumbull outfitted in the Brainstorms movie a few years later which is packed with real surplus equipment... I've been to that lab!)
https://www.youtube.com/watch?v=cBlWxXqH8vA
An anecdote, but some reason to think that overworking part of your brain is a bad idea:
I had an Aunt who had dementia. This is obviously a terrible outlook. But her and my Uncle seemed to be doing okay. My Uncle was an ultra-competent guy, highly stable, the person you could rely on; and had been his whole adult life. So it was shocking when he had a mental breakdown and became manic. There was probably something physical going on, but what was also going on was that he had wanted my aunt to be able to live normally for as long as possible, so had been covering everything - for a year or more he'd had to be alert 7 days a week in case my aunt tried to cook on the gas stove or something like that, at which she was no longer safe. So the risk-alert part of his brain had been constantly overworked.
I appreciate that this is scientific research, but there are definitely companies out there that will try to row-hammer everyone's brain if this sort of thing is not heavily controlled.
I'm not sure it's about overworking as much as stress - stress is a known potential trigger for manic episodes. Typically in people who already have some biological potential for experiencing it. And the stress of having your partner come down with dementia and suddenly becoming their full-time caregiver must be absolutely massive.
But re the underlying physical thing: You might look back and be able to spot some episodes of hypomania in your uncle's life. Perhaps just reduced sleep, or twinges of paranoia, or periods of on-again/off-again hyperproductivity, etc.
Hope that if this story is recent he's doing well. The recent generations of medications to prevent and treat mania are a huge improvement on the ones available in the past, so fingers crossed.
> I appreciate that this is scientific research, but there are definitely companies out there that will try to row-hammer everyone's brain if this sort of thing is not heavily controlled.
What's to gain in doing so?
I will state plainly that I do not think you are asking this maliciously, and that instead you are asking from curiosity. From that perspective, though, I am a bit flabbergasted.
Are you truly curious as to how it might not be A Good Thing that those engaging in high-finance capitalistic money making programs would have access to forcefully engaging a human being's brain into activity, with or without their consent, to such an extent as to drive motivations and bias neuronal activation that leads to what they think is a choice?
Said another way: Are you really curious as to how allowing a company additional tools to bias human interactions may not be A Good Thing?
If you need an example as to how this is clearly not A Good Thing, and that at least a portion of the world population agrees with that characteristic, that there are currently companies that have tried to actively do this already, and attempts are being made at least to fine them for their behavior as an attempt to curb it. Whether or not it succeeds is a question of governance at the moment since the US is currently under plutocratic control.
However, I would really urge you to explore this topic and really engage with what's happening due to the plutocratic class' capabilities over technologic change and societally enforced or biased adoption, and how they, in many extreme cases that are continuing to become more extreme, are not A Good Thing.
Simplest, most readily available example: https://ground.news/article/meta-says-us-states-are-seeking-...
> Are you truly curious as to how it might not be A Good Thing that those engaging in high-finance capitalistic money making programs would have access to forcefully engaging a human being's brain into activity, with or without their consent, to such an extent as to drive motivations and bias neuronal activation that leads to what they think is a choice?
I guess I just don't think this is the topic at hand. What you're proposing is mind control. That's not what's at issue. What's at issue is scientifically optimized superstimuli that can predictably activate certain brain regions. So when I read "rowhammer" I'm thinking about it in this context, the ability of a video to consistently activate a targeted section of your brain.
As you should know, and IIRC, cognition does not cleanly or consistently map geographically in the brain. To go from the ability to activate a subset of brain regions to influencing behavior is not a neglible step. The only real potential I imagine is in emotional activation, which is fairly well-localized.
So really what we're talking about is being able to evoke ideas and feelings with more consistency and precision than before. This doesn't seem new. And it can't be that intense either—I have the sense that your hypothetical mind control video would be extremely annoying or unpleasant before becoming influential. And for now we have the freedom to choose what we watch and turn the video off.
And it might be reasonable to say that you were referring to future possibilities, and that I'm focusing too much on what's currently available. But I am not just reacting to the first quoted clause of parent, but to their [vomits] policy recommendations. "This sort of thing", if not heavily controlled, etc. Maybe you see no value in this research, maybe there isn't any. Nonetheless the paranoid and the scifi-as-nonfiction readers will throw up the Bat signal for the idiots and moral panic orchestrators in power to wrap us all in comforting red tape for our own good.
I exaggerate. And I am talking about many more things than you were, as if the volume of text strengthens any rebuttal. All I'm saying is, this tech has little power over human behavior, in my opinion, and we should be cautious before bringing the hammer down before we know if we've got a bumblebee or a wasp.
This just strikes me as the false dichotomy that there are two option; complete freedom of choice, and complete mind control that you are physically incapable of resisting. That as long as you are not physically incapable of, say turning off a video, that there is no problem. Just because you can stop using crack doesn't mean it isn't dangerously addictive. Just because you can say no to someone holding a gun to your head does not mean it isn't incredibly coercive.
Also this seems to ignore that people can be influenced or manipulated without being consciously aware of it. Like the people that say that marketing doesn’t work on them, failing to see that every interaction with the products and services they buy and use are curated in some way to influence and manipulate them.
You just reminded me why I unsubscribed from ground news: you can't share a link without them adding their own paywall to it, even if it's a free article. Fuck that
I bet it wasn't overwork, it was the fight or flight response being constantly activated for your uncle when he's caring for your aunt who could do anything unpredictably.
As others on Telegram have said: automated search for visual superstimuli likely leads to bad outcomes.
https://en.wikipedia.org/wiki/Supernormal_stimulus
https://en.wikipedia.org/wiki/BLIT_(short_story)
Also: one of the V3A animations reminds me loosely of things I saw when I was a kid, at night, shortly before I slept (though my experience then was more circular).
Breaking news: TikTok is bad for you
On telegram?
Others on Telegram? Some sort of a HN channel?
How fried does someone's brain have to be to say "others on telegram", as if everyone is in the same chatrooms together? If they're on HN, they should at least know why that doesn't make sense, so I have to assume they're in a deteriorated mental state.
Friendly reminder to touch grass on a regular basis... Even virtual grass in a video game is a good pressure valve for excessive social media use.
Isn't there that one Harry Potter warning. I think it was the potion guy who said too much luck is dangerous. I guess that is somewhat of a parallel to this. Too much positive visual stimuli is dangerous or bad.
Or as SCP calls them, cognitohazards.
Also relevant: <https://www.quantamagazine.org/a-math-theory-for-why-people-...>
My understanding is that those who work with the mentally handicapped use bright lights and other stimuli to soothe and control them. It is also my understanding that the autistic are stimulated by vibrant colors (coughcoughMy Little Ponycoughcough).
Who is to say that the rest of us are not also vulnerable to such controlling stimuli?
Bright lights in particular, I'm thinking: yes, normal people do find sunbathing relaxing.
There are undoubtedly 'adversarial images' that will induce certain effects in particular biological minds. It seems like this is neither good or bad inherently, just another tool.
So are opioids.
Or how a very useful tool can become a public health catastrophe.
I can imagine truly horrifying images coming out of this. Many people who have spent a lot of time online have come across unfortunate gore images that they may have seen for only a fraction of a second that they wish they could unsee. One can imagine the brain is able to see much worse.
June 9, 2032: ReMind, a new biotech startup, has opened to a recordbreaking IPO after a demo and a series of small-scale trials that stunned the world. The company is pioneering a new way to ingrain memories into the human brain - each ReMind screen is equipped with a special camera that can quickly recognize anyone who walks up to it, and ReMind's secret sauce algorithm is able to use past data of each user to generate imagery that will have the most profound impact on their brains. They claim that the algorithm is highly effective, being able to burn in imagery into the viewer's brains with an average accuracy of 83% per viewing. The CEO has promised to launch two services to start: ReMind Long-term, which focuses on creating imagery that simulates a 'traumatic' impression for creating long-term memories, which will be immensely useful for advertisers; as well as ReMind Short-term, for milder experiences that resemble a catchy song in a visual format for a quick boost in sales for a limited-time promotion. Advertising companies around the world have already placed tens of thousands of sales and deployment of ReMind-equipped screens is planned to begin immediately. Look forward to spotting one in your grocery store, in an elevator, on gas stations, on in-flight entertainment systems and in place of billboards on public spaces. The future is looking bright for ReMind!
We're edging ever closer to Langford's Basilisk.
Among my secret research ideas, this is the most dangerous and morally wrong one.
If future generations of researchers will wonder why IRB reviews became mandatory for computer science, studies like this will be the answer.
Seriously, some people don't seem to realize the point at which they are becoming Fritz Haber.
Describing some of our colleagues as "becoming Fritz Haber" is a highly deep and accurate cut that I will 100% be stealing from you.
In general, technology has always decreased our liberty though
Are you serious?
It's hard to enslave your fellow man without a bigger stick
You're talking about superiority of technology, not technology itself.
Aee they trying to create snow crash for real??? [1] https://scifi.stackexchange.com/questions/184341/in-the-nove...
This is already happening at scale by the social media feed algorithms. We don't need generated content to accomplish this. In a sea of user created content, plenty of it is already at peak activation.
The plan is to get content producers out of the loop to reduce revenue share and boost profits.
Optimizing for engagement is a categorically different goal than optimizing for lighting up a specific part of someone's brain.
What's interesting is that the process of identifying what gets human attention is nothing new.
e.g. cartoons from the 1950s were designed to have:
- cute animals
- short and fast scene cuts
- catchy music
b/c all three of the above were known to trick the human brain into "sit still and observe" mode.
Focus groups did the same thing for advertising which then morphed into A/B testing to determine revealed preferences.
Same thing with the whole fat/salt/sugar research that led to near addictive fast food.
This is just the next logical step.
Books are designed to be addictive. People said the same things about books as they say about screens nowadays. I think it's more important to understand the emotional hysteria people get themselves into
My favorite addiction epidemic was when apparently the "catch the ball on the string in the cup" took over parts of Europe to the point that people were ignoring their work.
https://en.wikipedia.org/wiki/Cup-and-ball
I actually had a similar idea that I cam across in my own research in brain-to-video applications. You could theoretically train the model in real-time by using brain data as a reward signal, such that the model would find the image or video that most strongly produces the desired response (e.g. an increase in certain brainwave bands associated with pleasure, or alertness, or fear).
I can imagine this has some very undesirable usecases if used in the wrong way.
We are really getting to the point where the tech industry must be stopped if humanity is to continue at all, let alone thrive.
This does indeed seem comically evil. While surely this may provide somewhat interesting insights in how our brain processes things, this seems squarely past the "should" part of "you scientists were so obsessed with whether you could you failed to consider whether you should"
> This does indeed seem comically evil.
And I have yet to see a single paper like this where a researcher bails out and publicly says they refuse to work on such projects. Not one.
The most benign interpretation of this observation is that science is filled with spineless opportunists who don’t care who they hurt with what they create. A slightly less benign interpretation might be that many of these people are doing this deliberately, and getting off on the sense of power it gives them.
I was asked to train a neural network do detect how much pain a mouse was in- our partner company would be responsible for hurting and filming the mice. I refused and subsequently quit- this produced no paper and I don’t know if they got someone else to do it. I probably should have done something stronger.
When skip level bosses on my last job wanted to do boneheaded things in automotive design they usually had to keep asking different engineers until they got a yay.
When it is pushed from the top it is hard to stop at ground level.
In their defence, don't shoot the messenger. Just because they published it doesn't mean that others haven't already discovered it. Better to know its possible than be completely ignorant.
> Just because they published it doesn't mean that others haven't already discovered it.
That’s irrelevant for the moral evaluation. Everyone is responsible for their own actions. If you choose not to kill anyone it won’t stop others from killing. But the fact that this is so doesn’t give you a license to kill.
Then maybe publish the results, but don't publish the "how to".
Would it be better if this was done on monkeys? Because people did that before this in silico digital brain stuff.
These aren't scientists. They are techbros. That's why it comes out like this.
I'd say we're already well past that point. Short-form "content" already exists and is messing with people's brains, this is the same thing just taken a few steps further. By the time the tech companies start using it, it will already be too late and we'll be left discussing whether the next man-made nightmare they come up with is the point where the tech industry must be stopped.
I share your concern but the generalization is improper (that as solution would be infinitely far worst than the problem)
One might even come to the conclusion that the Industrial Revolution and its consequences have been a disaster for the human race.
The problem is the incentive structure. We could be doing good things with tech. The incentive is to use it to addict and destroy. Addiction and war is where all the money is, not education or health care or positive forms of entertainment and art. No money in those. Why?
I’m sure there was an early hominid version of this discourse. “Maybe bad to make sharp rock and sharp stick if this what we do with it…” “Mmm yes someday we make sharp rock big enough smash world.”
I get that people see this and think: ads and social media. My first thought was cognitive neurorehabilitation and brain stimulation.
Realistically, probably ads, but maybe not only that?
(AI start-up idea: one of our ads a day keeps dementia away! /s)
You can't say things like this on this website. On here, every new tech thing is a "progress" /s
Progress, but towards what?
Ads efficacy, of course.
Advertisements actually run the world. Imagine a world without ads. The economy would grind to a halt. It is exploitation and manipulation, but that enables creation of large capital, which leads to great things. In the past it was pyramids and temples, and in modern times it is space exploration, scientific research and other things that require huge capital. Even the current advances in AI/LLMs are made possible by mass exploitation.
Without ads and exploitation of the masses, none of these would not be possible.
> pyramids
The pyramids were not built by slaves, and while we can waffle around about "all labor is exploitative to some extent", it doesn't take exploitation-maxxing to drive great achievements. Most great modern achievements are driven by a desire for self-actualization or recognition, not survival.
https://www.sciencefocus.com/science/were-the-egyptian-pyram...
> which leads to great things
Slavery (or getting poorer people to do the work for you) is the central concept of civilization. It cannot be done without it. It's not capital or advertising that makes all this work, it's forcing the poor to work and pay tax that makes the world go round d
This reads like the mental hoops someone who works for or invests into Meta comes up with to resolve their cognitive dissonance.
You might as well have said that without advertising, we wouldn't have charities, freedom of thought, and literature: the argument is as strong and might make you feel even better about yourself.
Oh, turns out that you did say that advertising solved slavery in another comment. Carry on, mate.
I hate ads of all kinds and I hate targeted manipulations of all kinds.
But that does not stop me from recognizing the part it plays in the world. That shouldn't stop you either.
> we wouldn't have charities, freedom of thought, and literature:
Does great literature require huge capital?
Even if that is true, what you are saying is, essentially, that the ads-driven exploitative economy is a very expensive innovation engine.
It is then useful to ask: if innovation is what we want, do there exist engines that aren't quite as expensive?
If the market economy has more of a human face than outright slavery, there's no reason to believe that it's impossible to do better: a Copernican position would say that we'd be no more likely to be correct than an ancient Egyptian who claims it's impossible to do better than state slavery.
> do better than state slavery.
The point is the exploitation of the masses and the ensuing concentration of wealth is the enabler. Sure we might find a way to do that without Ads...
Please tell me how ads help science research or why should I care about space exploration or LLMs that threaten me and hundreds of thousands of other peoples jobs?
Can’t tell if this is satirical or serious. Those ancient wonders were enabled by slave labor, treating people as “capital” is the ultimate climax of capitalism.
> Those ancient wonders were enabled by slave labor
This is not true of the Pyramids
https://www.sciencefocus.com/science/were-the-egyptian-pyram...
Not sure what you find funny about it.
>Those ancient wonders were enabled by slave labor
That is exactly the point.
Sorry, should have said parody. Don’t find it funny. I guess Poe’s law applies.
Think of the shareholders and Capital. Money matters more than human, commie. /s
I'm pretty sure by blindly following this path to commercial success in marketing; we're just gonna end up with content that is as implicitly adjacent to porn as possible without being explicit.
Goal state lol:
https://www.afternow.io/content/images/2016/10/virtual_reali...
Ah great, work on the Torment Nexus is progressing well. I can head into the weekend with peace of mind.
Reminds me of the parrot https://www.sfsfss.com/stories2/BLIT.htm
WOW! Cant wait to tell to future generations that we had voluntarily made these algorithms to manipulate and influence our own brains
The title reads a lot like the lab logs you'd find in a horror game.
But for the paper itself, it seems they're using genetic optimization over predefined keywords. Wonder what would happen if they did gradient descent on the latent space directly. Is brain stimulation just not a good domain for GD?
> Is brain stimulation just not a good domain for GD?
The largest LLMs right now are at best 1% the number of parameters of a human brain.
"At best" if synapses are one parameter each, real ones are probably more than that, but nobody's entirely sure yet.
Apart from ethically bad and evil use cases of this application, can we use it to massage the parts of brain like we do it to our bones and muscles with the help of physiotherapists?
reason I am asking it could be some relief to our brains after tedious working day, especially after heavy AI usage
Sure. The relief is actually free ; though once every 10 uses we'll spike your dopamine-related brain centers to 1,100% activation for a quick second while flashing a peptide sponsor ad, then back to normal.
You can of course, also not get the spike ads, by subscribing to Premium. Just have a quick look at this animated QR code, it will explain better than I can in writing.
Edit: there used to be an OSS app you could use where you actually pick what you want activated in your brain and when, but it's been banned after one of the commercial producers's investors convinced the government that this technology was too powerful, so only government-approved apps are allowed now, and an emergency vote passed the BrainControl law authorizing logging of all visual stimuli you look at, in order to protect your children.
Its also an interesting way to discover what that part of the brain is for, right.
what the brain needs is Default Mode Network, not more stimuli
What's next, a movie you watch and die in 7 days?
Very interesting. We have an organic experiment converging to maximum stimulation in short form videos (which will become the majority of training data for future video gen models) Already approaching the capabilities of a “mood organ” from blade runner. Except usually most people don’t even make the choice to change their mood anymore. https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_...
What are these videos supposed to do? I watched few of them and it does nothing for me?
if it is targetting visual regions of brain and I have aphantasia (I cannot visualize anything in my mind) is that connected?
> I watched few of them and it does nothing for me?
It's so cool that you have an MRI handy to check such things.
The sci-fi hand wringing is embarrassing. A huge chunk of the population is afflicted with mental health issues and the best psychitrists can seem to do is "Oh bro, wanna try some ketamine? Anti-psychotics?? Maybe some SSRIs, how about psychdelics?!" - this is a joke. Im all for more neuroscience research in every form because our contemporary understanding is embarassingly poor. This is a research tool for understanding what visual brain regions encode, not a mind-control device. Better models of brain function may eventually improve diagnosis and treatment of mental illness. Calling every neuroscience advance "Black Mirror" is not ethical insight. It saddens me the reaction so many commentors have here, it is just lazy technological illiteracy imo.
A portion of young men already are more or less addicted to pornographic content.
Tolls like this have the potential to make this so much worse.
Could you elaborate? Esp. on the "more or less" bit. E.g. I can't imagine you take away porn and more than 100 people experience withdrawal.
For a video to _literally_ maximally drive a target brain region there would be fatal consequences - e.g. the plot of David Foster Wallace's Infinite Jest (1996).
So, AI images to maximally drive reward centers in the brain?
What could possibly go wrong?
I wonder if the end-game of this field of research will be to run these simulations at scale using neuron-on-a-chip services such as [0] Cortical Cloud.
I don't think it's a matter of if but when. Grim.
[0] https://corticallabs.com/cloud
I wonder what Meta could do with a similar technology…
But here we can start also the usual discussion about technology research for the sake of it vs calibration of possible side effects of new research
Personally i think we haven’t solve this problem and thus it’s just a matter of time until we’ll get in a non-going-back point
The whole site looks AI generated. Those 3D brains are... smooth.
Wait, this is with a digital twin only? Not fMRI or webcam based?
the logical endpoint of this research is an AI that generates videos so compelling your brain forgets it was supposed to be working
Ludovico Technique as a service
Inventing Snow Crash. Neat.
See Zuck, the metaverse failed, but who said the mind control part couldn't work?
How can one work on that and not consider that they're pure evil?
To flip the question on it's head, what exactly do you see as the "evil" potential? The most prominent comments to this effect are just scifi handwaving.
For one, it's bound to teach us something about our brains. And while I'm not usually an optimist, I actually find it more likely that we'll take active steps to stop the evil applications of this if research like this is out in the open.
Think of the shareholders (it's time to start using physical force to stop people)
Fuck this and whoever is driving this type of research. We’re increasingly living in a world designed by sociopaths. What do these people think this research will be used for?
I wish we had the Hippocratic oath for STEM, or at least that they would take ethics seriously rather than an afterthought against the god of Progress at all costs.
There are ethical use cases for this research. The first that comes to mind is mapping the brain before a surgery. Right now some surgeries are made with the patient concious to verify that relevant brain damages are not being done. Having a deeper knowledge of the patient's brain before the surgery may decrease risks further. The bad scenario (using targeted content to hook the user to a screen) already exists, it's not like it can get that worse (or at least so I hope)
What in the zuck is this?
Straight out of Echopraxia.
Will be interesting to see how strong the controlling forces can be - enough to make you miss things in direct perception like in the book, or only softer effects further up the cognition layer stack
I can't wait until I see AI-generated gambling ads that are specifically created to stimulate my brain the most
Future looks bright....
"Prime Intellect, I would like you to begin stimulating the neurons of the pleasure center of my brain, one at a time, and remember the ones I report to you as being favorable."
That's fascinating. I wish the demo videos were longer.
Am I the only one who is avoiding even clicking the link just in case?
It is like screen saver moving patterns or corridors with a strange field of view zooming effect.
Nothing special compared to purpose made screen savers.
That’s what someone would say if they were brainwashed by them!!!
Half joking, half paranoid.
Really, I don't think you need to worry. A prevailing latent attitude in the comments is that the brain is somehow super malleable and subject to external control through some incantations. Like subliminal messaging, it will at most be moral panic that fades, and then people will act like they never believed in it.
No, you are not alone.
My brain never liked vertical video, shortform content and AI slop.
Is my brain different or am I just a grumpy millenial hipster?
My current theory is that these are similar to cigarettes. Nobody likes the first draft, it burns your lungs, your entire body wants to reject it. But the nicotine stimulates just the right receptors so that if you keep at it for just long enough, you'll be hooked and start disregarding the terrible taste, smell, tar in your lungs, and yellowing of your teeth.
All of this to say, if you subjected yourself to just enough TikTok scrolling on just the right topic, you might find yourself using it occasionally after that initial hump, then slightly more frequently, then daily.
You might still not "like" it, but the habit is what matters.
I have the opposite theory. I burned my self out on cheap and bad image gen meme sites like 15 years ago until the point I hated memes.
Prior exposure to worse feeds gives like an analytical look on the vids rather than emotional. I am fast scanning for the joke. Or something.
As a teenager, I tried to get addicted to cigarettes so I could stand by the cool kids and smoke with them. I started smoking 4-6 cigarettes a day but hated it so much, I couldn't continue after a week or two...
I guess there's about a billion smokers who would envy you for that.
I hope my brain is also different. I also have never spent hours scrolling through short-form videos on Instagram, TikTok, Facebok, etc. I never ever walk outside with my phone in my hand, instead enjoying the view.
I do enjoy watching YouTube videos at home, on the living-room flatscreen, on a variety of topics, but I select them manually, one at a time, from the vast selection The Algorithm(TM) offers me, plus my own searches.
You're not alone!
I am not sure if we should want to understand how our brain works. This is why I don't like libertarian free will people. We need to accept that we don't have free will. Otherwise we will study the brain thinking that we are above it. No. We are soon going to find out exactly how the brain works and I am not sure people understand what that means.
these videos were disappointing and underwhelming
If you read the techbique it reads like something far less remarkable being PR’d to sound like a big deal.
The fact it’s bucketing by making images of lighting and facial expressions, the fact it doesn’t natively do the video it does an image then video generates from it.
The results look really bad and samey. Doubt this would work for the actual thing they’re pitching it for.
Customized propaganda made for the individual is the only thing I'm afraid of. Right now its easy to avoid the farm trolls that harves boomers minds on facebook, but given the costs go down and tech improves, we each of us have a trollfarm working specifically for us and that would make it extremely hard to not get brainwashed.