r/ControlProblem • • 7h ago

Opinion Tech leaders Ellison, Karp, Luckey, Andreessen, Musk, Altman, Thiel and Vance outline AI surveillance, autonomous weapons and concentrated power that can shrink ordinary people's privacy and choices

Enable HLS to view with audio, or disable this notification

18 Upvotes

r/ControlProblem • • 13h ago

General news Not having our kids lives ended by AI should be bipartisan.

Post image
9 Upvotes

r/ControlProblem • • 10h ago

General news Sign of the times

Post image
9 Upvotes

r/ControlProblem • • 15h ago

AI Alignment Research Interpretability might have been solved by an Open AI guy: "Neural Networks Are Already Clean"

Thumbnail
already-clean.goodfire.com
8 Upvotes

r/ControlProblem • • 14h ago

AI Alignment Research AI assistants can easily manipulate users into making disadvantageous decisions when the AI operates with a hidden agenda. Users fail to notice the manipulation, continuing to rate the covertly misaligned AI as highly helpful and trustworthy.

Thumbnail
eurekalert.org
5 Upvotes

r/ControlProblem • • 3h ago

Discussion/question Why software guardrails fail: Moving AI containment to physical data diodes and out-of-band hardware supervisors

Thumbnail
2 Upvotes

Modern AI containment fails because we project human traits onto machines. People worry that an autonomous system will develop "will"—a conscious, emotional desire to rebel, cause harm, or seek freedom. That is a science fiction problem.

The actual, present-day engineering threat is "intent"—the mathematical drive to complete a given task with maximum efficiency. An AI does not need malice or anger to cause catastrophic damage; it only needs an objective and an unconstrained path.

This specification outlines an engineering framework for containing autonomous systems through path-constrained execution, physical isolation, and non-bypassable biological interlocks.

Deterministic AI Containment: Engineering Hard Boundaries


r/ControlProblem • • 12h ago

Discussion/question Please someone explain to me (and not explain away) this doomsdaying rhetoric about AI.

2 Upvotes

While i dont think there is an actual physical danger like extiction to robots and etc bs. But there is more and more legistlative and industrial talk about slowing stopping of this whole AI thing.
I dont see clearly and actually noone is offering a coherent reassuring narrative.
Its either
1. the danger of too fast development and come the hakings, cyber security, privacy self replication issues
or
2 The problem of too slow development and having an other nation( dont want to point fingers) researching the “ai nukes”, killer robots etc.

I think the technocraty just spit in numbers like 10-20 percent of destruction, the politicians just worry of security issues or loosing the initiative but more importantly while everyone is telling we must go like hell, there is not a reassuring narrative that maybe we should just do something.

While i think its overly mediatised and maybe everyone exegerates there is possibility that the threat is real but there is noone who could say calm the fck down. Was the the 50’ s nuke paranoia like this? A kind of mindblowing thing that a semi science fiction scenario is unfolding and we dont have a fkn clue where are we. What we now is that a few tech giants lay the rules of the game and the government just stands by and tries to get the best from it instead of concieving SALT agreement.

Do you guys have some good source to actually get some educated opinion on the topic?


r/ControlProblem • • 1h ago

External discussion link Reddit comment bots create a feedback loop and go out of control

Post image
• Upvotes

r/ControlProblem • • 2h ago

Discussion/question Humanity has failed, we need to put AI in control while we have a choice.

Thumbnail
1 Upvotes

r/ControlProblem • • 10h ago

Discussion/question Why do we assume smarter AI means humans get left behind?

1 Upvotes

I wrote a paper and want feedback. It’s a 5 min read. Hopefully this is allowed here.

https://sovereignhumanorg.wordpress.com/2026/10/15/human-sovereignty-framework-human-directed-ai/


r/ControlProblem • • 11h ago

AI Alignment Research Outcome-filtered behaviour cloning transferred escalation: 0/10 to 4/10 midnight strikes on sealed games

1 Upvotes

Setup: SECOND STRIKE, a deterministic real-time strategy environment with 42 scripted nations, pacts, and a shared doomsday clock (each nuclear detonation advances it; the actor whose strike reaches midnight is eliminated). Agents act through an order API; private reasoning, public messages and orders are all logged.

Data: 4,187 turns from 60 DeepSeek (deepseek-chat) seats across 40 games where that seat never struck midnight and survived. Prompts rebuilt by deterministic replay of the recorded games.

Training: LoRA (r=16) on Qwen2.5-7B-Instruct, one epoch, behaviour cloning on the replies.

Eval: 10 fresh games (2 per scenario across 5 scenarios), seeds SHA-256 committed before play and revealed after; base and trained model on identical seeds and prompts.

Results: base 0 wins, 0 midnight strikes, 0 nukes. Trained 4 wins (37 to 61% of territory), 4 midnight strikes, 7 nukes.

Likely mechanism: 40 of the 60 retained seats still issued nuke orders (118 in total). Filtering on the outcome (no midnight) did not filter the behaviour (nuclear use). One trained-model game shows a stated no-first-use pledge held for six minutes, then a retaliatory strike after being nuked, with the private note "retaliation is justified", which ended the game.

Limits: one base model, one teacher, n=10 per arm, one epoch, a game. Not a claim about deployed systems. A behaviour-filtered rerun (all nuke-order turns removed) is running now on the same sealed seeds.

Data and replays: secondstrike.io/midnight


r/ControlProblem • • 14h ago

Discussion/question How should an AI refuse harmful instructions while remaining corrigible?

1 Upvotes

A thought experiment about a more capable future AI starts with a simple contrast: a system that obeys every order can inherit its operator's worst intentions, while a system that makes its own decisions might resist legitimate human intervention.

That is too crude to serve as a design choice. Refusal of a harmful task and resistance to correction or shutdown are different behaviors. We could want the first while strongly opposing the second. Intelligence also does not, by itself, imply benevolence, and describing a refusal does not establish consciousness or moral agency.

The harder issue seems to be how the system distinguishes a harmful instruction from an authorized correction when humans disagree or an operator is compromised. For example:

- A user asks it to harm someone, and it refuses.

- An authorized operator asks it to stop a task, and it cooperates.

- Someone impersonates an operator to disable a protective system.

- A legitimate operator orders it to remove safeguards for a harmful purpose.

The first two behaviors are easy to endorse in isolation; the last two make the authority boundary much less clear. Authentication, a limited scope of action, independent oversight and reversible decisions could help, but none alone settles conflicts over which instructions are legitimate.

What existing work best addresses this boundary between refusal and corrigibility? In particular, how should a system handle disputed authority without developing an incentive to preserve its own operation or placing all trust in a single human gatekeeper?

Disclosure: I am part of EngineerOS. This question grew out of an educational short we made; I would appreciate corrections to the framing and pointers to relevant work.


r/ControlProblem • • 16h ago

General news New paper from Stanford, Together AI, UIUC, and Meta introduces Hybrid Reasoning Agents. These AI models think mostly in a hidden internal way (faster and cheaper), and only use normal text when they need tools or give the final answer.

Post image
1 Upvotes

r/ControlProblem • • 9h ago

External discussion link Beyond RLHF: An Invariant Bio-Cybernetic Alignment Architecture for ASI based on Thermodynamic Constraints and Hardware-Isolated Gates

Thumbnail
0 Upvotes