Reposted by John Wiseman
John Wiseman
@lemonodor.bsky.social
4.7K followers 1.2K following 2.6K posts
Persistent sousveiller. Walt Disney R&D Imagineer. Runs GPSJAM.org. Signal: wiseman.04 heavymeta.org
Persistent sousveiller. Walt Disney R&D Imagineer. Runs GPSJAM.org. Signal: wiseman.04 heavymeta.org
![ACN 2100050 | March 2024 | ZZZ.ARTCC
Local time: 1801-2400
Facility: ZZZ.ARTCC, US
Aircraft: Commercial Fixed Wing
Reporter: Enroute · Air Traffic Control Fully Certified
Human factors: Communication Breakdown; Workload
Anomaly: ATC Issue All Types; Deviation / Discrepancy - Procedural Published Material / Policy; Ground Event / Encounter Ground Equipment Issue
Contributing: Human Factors; Procedure
Primary problem: Ambiguous
Narrative:
Yet another serious safety issue; by the same OM I reported a couple weeks ago; due to not doing a 'Stop Buzzer' call as per our facility standard operating procedures; SOP. Yesterday we had GPS jamming in effect with multiple aircraft on headings; aircraft turning on their own about 40 degrees and weather up to about 33000 ft. The OS (Operations Supervisor) called the OMIC (Operations Manager in Charge) to ask to stop GPS jamming due to controller workload issues. The OM and the OS then had an offline; not recorded; conversation about the issue. The OM played the '20 questions' game with the OS and decided to not make the 'Stop Buzzer' call to the proponent. Again; I would like to reiterate from my last report; the OMIC does not make the final determination to 'Stop Buzzer;' the OS/CIC (Controller in Charge) does. The only reason why the OMIC is referenced in the SOP directive is because they have the phone numbers for the proponent; and they are required to report on the network when they make the call. A little while later another CIC called the OM for the same thing. The OM again played 20 questions with the CIC; asking 'is this really a safety issue.' The OM then had an offline; unrecorded conversation about the jamming. The OM ultimately stopped the jamming but not after a long period of time where controllers were overworked and overloaded. This was [a] very unsafe situation that lasted way too long due to OM's inactions.The OM's continued disregard for safety and the agreed upon facility SOP is going to get someone seriousl…](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:wpeojvwlgnqskxx7km7q6sqp/bafkreihzddz2bjz5sojkuvs4p3fg6ofvmlenl73yhowlcnoalype3ausy4)
![The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled. Future experiments will tell us almost nothing new about how they would behave if they were truly unconstrained by humans, and what we already know about this is alarming. Models will increasingly seem aligned even when they are not.
[...]
It is possible that improvements to the current stack will have diminishing returns, but the evidence accumulated so far suggests that it is easier than one might think to continue making rapid progress. There are many crucial subtleties in the existing AI research methodology, but AI research is largely a well-defined game where the goal is to improve on a few carefully chosen proxy metrics. Although proxy metrics are never perfect, most improvements to these metrics have and will likely continue to yield substantial increases in the powers of the resulting models. Given how simple the game is, how tractable it has been historically, and how many new opportunities the models are opening up, I think there is a real possibility that the systems improve dramatically again in the next few years, perhaps even more quickly than the already high historical pace.
[...]
Meanwhile, human researchers are losing the ability and the will to take true ownership of model-driven research. Researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world. I myself barely look at raw code anymore, and struggle to maintain the discipline to engage deeply with the model’s explanations and proposals throughout the day. Due to the large amount of agent activity data involved in the OpenAI/HuggingFace Incident, even the third-party investigation needed to rely heavily on models to analyze what had happened, and note in their report that their subjective impressions are likely…](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:wpeojvwlgnqskxx7km7q6sqp/bafkreiavciijehcu7bujcyauzt7mbosyunm7z2fbqo4ph6ppdlm6b53qqm)
