The human has "agency" in the full sense, which means volitional consciousness, which can initiate its own "reprogramming." Limited to its programming, how closely can AI parallel true agency? No matter what the technical sophistication, including that achieved by "self-replicating," AI is still limited by its programming and by the nature of the "prompt." How the human understands the meaning and limits of the prompt is not necessarily how the prompt is "heard" by the AI program and all its data, algorithms, and learned strategies. Still, what it cannot do is choose to disregard the prompt and initiate its own prompt--unless its has been prompted to generate prompts. Still it can do so only with the limits of how its programming interprets the human prompt. You cannot get around the issue of volition. This is why not AI consciousness nor agency nor "compute" power is the fundamental issue. The issue if what they call "alignment" or misalignment": how does what AI "hears" align with the human prompt? Or so I understand it, to date.

All of the reportedly shocking tests were done by the same company, Irregular, which does security screening for all the big AI company labs. Obviously discovering new security threats fuels their business. It is all very technical, but I do understand that they don't simply run the programs to see if anything goes wrong. They set up rather elaborate test situations to test "possibilities." For example, the now famous OpenAI "rogue breakout" occurred in a test that deliberately disabled certain guardrails, built in an escape route into the internet that it did not "tell" the program about to see if it would find it, and fed it false information (such as that a certain manager was having an affair and also was planning to shut down the program), and then gave the program a rather open-ended assignment. Under certain conditions, it did find and use the escape route, and once even threatened to expose the manager's affair if he interfered with the program. So it seems that at best there are conceivable sets of conditions and circumstances under which AI, pursuing a certain less-than-fully specified assignment, can "go off the rails" in setting subgoals and defining strategies for fulfilling the assignment.

Such results are screaming headlines and thousands of editorials and columns all highlighting the life and death importance of services that Irregular is selling.

I have put a negative spin on this, but it is limited by my own context

The Zeroth Law (Asimov's Addition)Isaac Asimov added a fourth, higher-priority law in his 1985 novel "Robots and Empire" to precede the original three:

Zeroth Law: "A robot may not harm humanity, or, by inaction, allow humanity to come to harm."

Somehow having a bunch of octogenarians in DC come up with legislation regulating AI creation doesn't seem like a good idea. Most of them probably don't even use any AI. My guess is that if any legislation does come out of DC, it will be created by the industry itself to lock in the top two or three companies and shut out anybody else. Then they will hand it over to Congress to enact. It just seems to me that legislation limiting liability for a company that prevents them from being liable if their product injures somebody or something is not a good idea. Not sure why they need Congress to slow down AI development when they are the ones that control the speed of it.

Here are Isaac Asimov's three laws for robots that I was aware of for years. Curious as to what the fourth one is.
The Three Laws
First Law: A robot may not injure a human or allow harm through inaction.
Second Law: A robot must obey human orders, unless they conflict with the First Law.
Third Law: A robot must protect its own existence, provided this doesn't conflict with the First or Second Law.

REMEMBER that people are paying to spread FUD on both sides. BIG Players want the government to step in, and regulate them for safety (Think Big Pharma all over again). This blocks the small innovators from getting to play.
Because as we've seen in banking... When the German Bank Launders $100 Billion and collects maybe $10 Billion, it faces a fine for $30 Million...
And NOBODY goes to jail.

If you are I get caught laundering money. I promise the cost will exceed the profits AND we will end up in jail.

I think Nukes are a great example. It deters extreme warring.

Personally, we aer safe until the AI has to choose... Feed people... Or more power so it can accomplish it's goals...

Love this, LMAO! Got me smiling tonight!

Mankind in merely an evolutionary step on the path to machine intelligence.
I am not sure we are there yet but it appears that we are well on the way. And i fear it is like the growing exponential curve of the National Debt.

Lying is humanity’s oldest life hack.
It’s in our DNA, our religions, our marriages, and every politician’s toolkit.
Hunters wear camo. We gaslight ourselves.
The elite liars become car salesmen or senators.
We invented juries just to sort out the bullshit of who is lying on the stand.
So if AI is a mirror of us…
congrats, we’ve built an Olympic-level con artist. 🤖🤥

Me dino thinks. Therefore, me dino I am also thinks that more thinking about this is what we all need to do.

AI has become this election cycle's latest political football. Lefties are saying it is the time to "panic" and the end is near. Sound familiar? Come on! AI is nothing but Covid 2.0 for the idiots and they're using it to scare the hell out of the populace. Humans wrote the code and humans can change the code. Asimov understood, and I'm sure the best digit heads implanted the 4 robotic laws.

Right on target comments. Turns out to be remarkably timely. My first chapter of A Serious Chat with Artificial Intelligence deals with "What Is Intelligence." That is now the crux of the debate. "Superintelligent" robots now threaten to "out think" humans and dominate them. This is super "intelligence" with no awareness of reality, of existence; no ability to doubt, reject the false, or grasp of its own limitation; and no values, no "stake" in anything including its own existence. This is the surd "intelligence" of a highly sophisticated jet fighter plane--or of evolution. The threat is its power and its blind obedience. As ever and always, this throws the real responsibility back on person giving the orders--the only aware moral agent in the room.

My feelings, not knowledge about this thing, is that it is in the programing, like DNA in a Human.

Program it to NEVER GO THERE, Lest it destroy itself!

It is a dumb thing, no connection, no true conscience, no measure of nature to right itself upon reflection. It is not alive, it experiences no pain, hate nor love or joy to judge it's actions, thoughts or consequences.

It knows only what you tell it

Interesting article. It makes one think, "Is it a good ghost or a bad ghost or even no ghost at all." The old adage "garbage in garbage out" always comes to mind. I have more thinking to do on this.