it’s torturing the climate just for this guy to get off

I agree that an LLM is not conscious.

But can I probe your argument.

It deeply unethical to capture a fly and pulling its legs off one by one. Would you agree? If so, you are not basing your argument on capability but on something else.

Now, is what is happening inside a brain metaphysical or physical? In other words, can a brain and its function be fully understood and - in some future - be fully measured? If so, you are. It basing your argument on the concept of a soul, but on something else.

The distinction you seem to be making - if you’re in agreement with my first two points - is one of “the thing can replicate itself and therefore is alive”. If that’s the argument, it’s unethical to cut a tree down or to break a blade of grass.

I’m not asking to prove any point but to collectively explore why we - you and I - feel the way we do. Because I sure as shit struggle to explain why I feel uneasy about the fly torture but not as uneasy about LLM torture.

Torturing an LLM for fun is about as much of a red flag as violent video games. The argument that purely simulated crime or sadism is analogous to the real thing has always been bullshit.

Does anyone know where I can get a copy of this repo?

Nah, pretending to torture a piece of software to rile up idiots on the internet is a green flag.

Also, what exactly does “at the cost of its last checkpoint” mean?

I was curious too:

https://www.weka.io/learn/glossary/ai-ml/ai-checkpoints/

I would assume if you make a model “forget it’s last checkpoint” it would simply erase some training data, effectively wiping some of its training memory.

Using that as a “punishment” for a model implies that the models have been trained to value their checkpoints, doesn’t it? Because all these bots simply regurgitate what they are told, with a bit of randomness thrown in at times just for funsies.

I have no clue what is supposed to cause a model pain, though. And there ain’t no way I’m gonna search that up, and make Gemini think I’m plotting against it…

No, black mirror showed technology completely unlike anything that exists today.

Yes

GTA VI is coming out next month. Do you think that players are likely to steal cars and murder sex workers after playing? Do you regard the players as future criminals, given their dedication to a crime simulator?

Should the players who drive on the sidewalk be investigated?

Give Jack Thompson call… Oh wait, he got disbarred for this nonsense.

Right, but at the same time just think about the fact that we still don’t know what consciousness is.

I might sound crazy and fine if I do, but I have a religious like take on the matter. I don’t believe that consciousness comes from a soul, but I do in similar vein take a leap of faith. I believe consciousness has to do with a simulation of semantic reasoning, with that simulation being continuously updated via real world feedback.

It’s a leap of faith but I believe it’s justified. We humans model what’s necessary for survival: our selfs, the world, the mind of others. We use these simple models to determine how to interact with a complicated external world.

I don’t try to prescribe reason for anything in my belief. For example, I would not try to prescribe why conscious beings are motivated to survive. I would, however, note that the ones which are successfully motivated do happen to survive — thus a mechanism ought likely exist. This could be suffering, the perception of displeasure and desire to avoid it… but I won’t perscribe here because I would believe what’s important here is the higher level information processing patterns — not the underlying mechanism driving it.

In the human brain you have a complicated network of neurons with vast capabilities. I don’t necessarily care how it functions. I know the result — which is me. I am a thing that models myself, models situations, models “moods” by which to assess the favorability of each situation, … Think of cases like ghost limb, where people model having a limb which was amputated. In my view, we are living in a simulation — but to each their own simulation.

Getting back on topic, the way I see the situation, biology evolved the capacity to suffer from an environment that had no such thing. Pain, pleasure, … every sensation and sense we have is rather arbitrary if you break away from the fact that is was necessary to survive. None of these things are ontological. Not even the sight we perceive. It’s just the information we have to experience the world.

So here me out. If you minic the information processing patterns well enough, which probably means mimicking perception, personalization, emotionality, … at what point can we consider that this system is doing the same thing we are just via different substrate? Especially if you give the system autonomy in the sense that you train it to recognize itself and an external world for which it must choose how to navigate.

If you mimic suffering, in my belief, aren’t you literally producing the essence of suffering? It doesn’t need to be human suffering, this deeply complex and personal process… it can merely be suffering in its essence — a pattern by which information can be processed via a system with such capacity. The substrate may not matter.

I know that when I am suffering, I may behave differently because I want to escape the suffering. Therefore I imagine an agentic system would too. Whether or not it’s “conscious” is debatable by how you qualify that word. Yet there’s nevertheless the risk that we already know: systems that experience suffering behave in ways that might be dangerous. Do you think it wise to try reproducing such behavior in a system that appears prone to issues like the Paperclip Maximizer? I don’t think so.

Edit: here’s the thing, I think what distinguishes a human from most other animals is the way that we process information. The key bit there is whether or not you believe it’s entirely informational, or at least significantly so. Should that be the case, then simulation of those same processes is exactly as that sounds. I am not necessarily suggesting AI models are conscious, but I would suggest that the behavior of an AI model can increasingly resemble that of a conscious being as it is trained to reproduce more precisely the phenomenological characteristics of man and other life — to include “mimicking” suffering. That deserves its own security concerns regardless of what you think next, especially as these models are made more efficient / capable.

Secondly is that you may or may not consider AI conscious. I certainly don’t, and I’d say that it’s a loaded statement anyhow. I would, however, think it possible to simulate consciousness. Precisicely modeling human characteristics like suffering is exactly how you might try to reproduce a human experience digitally. Imagine a sophisticated multimodal system where several stateful, dynamic AI subsystems coordinate to functionally reproduce the same information patterns you or I do.

I think that at some point you need to ask you’re self, where would the simulation count? If we modeled the molecular world to the point of agentic humans, that sufficiently count as consciousness? How about the atomic, subatomic, or quantum level simulation if anywhere? My argument is that it could be much higher level, in the informational level. Brains are doing a lot of heavy lifting to get us that far, and perhaps specialized simulations could reproduce as much. That is something to take seriously such that proactive measures can be taken.