Skip to content
Tiatra, LLCTiatra, LLC
Tiatra, LLC
Information Technology Solutions for Washington, DC Government Agencies
  • Home
  • About Us
  • Services
    • IT Engineering and Support
    • Software Development
    • Information Assurance and Testing
    • Project and Program Management
  • Clients & Partners
  • Careers
  • News
  • Contact
 
  • Home
  • About Us
  • Services
    • IT Engineering and Support
    • Software Development
    • Information Assurance and Testing
    • Project and Program Management
  • Clients & Partners
  • Careers
  • News
  • Contact

OpenAI touts new approach to AI safety after troubling executive transfer

OpenAI announced on Wednesday a new approach to AI development, one that it said “aligns models to behave safely without extensive human data collection,” although some have raised concerns about AI interacting with AI.

The move coincided with an announcement to address questions about safety priorities, following the reassignment of a key safety executive. 

On Tuesday, OpenAI Chief Executive Sam Altman said that AI safety leader Aleksander Madry is working on “a new research project”, according to a news report from Reuters. Other reports, including one from CNBC, said that Madry was being reassigned “to a job focused on AI reasoning.” Madry was among the executives that OpenAI announced in May would be improving the company’s security and safety efforts.

It also came at a time of renewed US government agency and congressional questions about the vendor’s dedication to safety and protections.

Ashish Thakkar, an AI programmer based in Mumbai, said the timing of the move was alarming.

“Something feels fishy about the whole thing, because just last month, OpenAI whistleblowers filed a complaint to SEC stating that the company does not allow them to speak openly about the safety concerns related to their AI technology. Is it possible they are re-structuring the entire AI safety team because of this?” Thakkar asked. “What I think could be going on is they are reshuffling the entire safety team, a team which would be led and controlled by a close group of people so that no leaks, or instances like June 2024 happen again. This is why there needs to be AI regulations in the US and worldwide now and not later. EU and China seem to understand this.”

When CNBC asked OpenAI about the move, an unnamed spokesperson said, without elaboration, that “Madry will still work on core AI safety work in his new role.”

Various industry players said that they were concerned that the move might be another indication that OpenAI’s focus is veering away from safety and data protection. 

“While it’s impossible to know exactly what’s going on behind the scenes at OpenAI, shifting crucial personnel from a safety role to a job focused on reasoning, innovation, and implementation says one of two things: He wasn’t effective in his previous role, or Open AI is shifting priorities toward innovation, potentially at the expense of ethics,” said Brian Prince, CEO of TopAITools.com.

Rob Rosenberg, a New York entertainment attorney, said that he is also concerned. 

“The re-assigning of one of OpenAI’s top safety executives from his role feels like a continuation of this pattern we’re seeing from OpenAI, where they announce initiatives towards safety and then undo those initiatives. We’ve already seen two of OpenAI’s senior leaders, Ilya Sutskever and Jan Leike, leave the company in May, citing issues over safety culture,” Rosenberg said. “OpenAI has been anything but Open.”

“Sam Altman has not been very forthcoming with OpenAI’s plans, including his recent post on X where he says Aleksander Madry is being reassigned to a new project, but does not disclose what that new project is,” he added. “An arms race is taking place among these generative AI companies to keep rolling out newer, better and faster products, and it feels like safety is repeatedly taking a backseat to those other initiatives at OpenAI.”

On Wednesday, OpenAI introduced what it said was a “new method leveraging Rule-Based Rewards (RBRs) that aligns models to behave safely without extensive human data collection.” It also published a technical document exploring the method in more detail. 

The company said that it introduced the new approach to address some weaknesses in its current efforts. 

“To ensure AI systems behave safely and align with human values, we define desired behaviors and collect human feedback to train a reward model. This model guides the AI by signaling desirable actions. However, collecting this human feedback for routine and repetitive tasks is often inefficient. Additionally, if our safety policies change, the feedback we’ve already collected might become outdated, requiring new data,” the vendor said.

“Thus, we introduce Rule-Based Rewards (RBRs) as a key component of OpenAI’s safety stack to align model behavior with desired safe behavior. Unlike human feedback, RBRs uses clear, simple, and step-by-step rules to evaluate if the model’s outputs meet safety standards. When plugged into the standard RLHF pipeline, it helps maintain a good balance between being helpful while preventing harm, to ensure the model behaves safely and effectively without the inefficiencies of recurrent human inputs. We have used RBRs as part of our safety stack since our GPT-4 launch, including GPT-4o mini, and we plan to implement it in our models moving forward.”

However, the company acknowledged potential drawbacks. “Shifting safety checks from humans to AI can reduce human oversight of AI safety and might amplify potential biases in the models if biased models are used to provide RBR rewards,” the statement said. “To address this, researchers should carefully design RBRs to ensure fairness and accuracy and consider using a combination of RBRs and human feedback to minimize risks.”


Read More from This Article: OpenAI touts new approach to AI safety after troubling executive transfer
Source: News

Category: NewsJuly 24, 2024
Tags: art

Post navigation

PreviousPrevious post:RFProposal vs. RFPartner:どちらが最適か?NextNext post:SAP restructuring to impact more jobs than expected

Related posts

휴먼컨설팅그룹, HR 솔루션 ‘휴넬’ 업그레이드 발표
May 9, 2025
Epicor expands AI offerings, launches new green initiative
May 9, 2025
MS도 합류··· 구글의 A2A 프로토콜, AI 에이전트 분야의 공용어 될까?
May 9, 2025
오픈AI, 아시아 4국에 데이터 레지던시 도입··· 한국 기업 데이터는 한국 서버에 저장
May 9, 2025
SAS supercharges Viya platform with AI agents, copilots, and synthetic data tools
May 8, 2025
IBM aims to set industry standard for enterprise AI with ITBench SaaS launch
May 8, 2025
Recent Posts
  • 휴먼컨설팅그룹, HR 솔루션 ‘휴넬’ 업그레이드 발표
  • Epicor expands AI offerings, launches new green initiative
  • MS도 합류··· 구글의 A2A 프로토콜, AI 에이전트 분야의 공용어 될까?
  • 오픈AI, 아시아 4국에 데이터 레지던시 도입··· 한국 기업 데이터는 한국 서버에 저장
  • SAS supercharges Viya platform with AI agents, copilots, and synthetic data tools
Recent Comments
    Archives
    • May 2025
    • April 2025
    • March 2025
    • February 2025
    • January 2025
    • December 2024
    • November 2024
    • October 2024
    • September 2024
    • August 2024
    • July 2024
    • June 2024
    • May 2024
    • April 2024
    • March 2024
    • February 2024
    • January 2024
    • December 2023
    • November 2023
    • October 2023
    • September 2023
    • August 2023
    • July 2023
    • June 2023
    • May 2023
    • April 2023
    • March 2023
    • February 2023
    • January 2023
    • December 2022
    • November 2022
    • October 2022
    • September 2022
    • August 2022
    • July 2022
    • June 2022
    • May 2022
    • April 2022
    • March 2022
    • February 2022
    • January 2022
    • December 2021
    • November 2021
    • October 2021
    • September 2021
    • August 2021
    • July 2021
    • June 2021
    • May 2021
    • April 2021
    • March 2021
    • February 2021
    • January 2021
    • December 2020
    • November 2020
    • October 2020
    • September 2020
    • August 2020
    • July 2020
    • June 2020
    • May 2020
    • April 2020
    • January 2020
    • December 2019
    • November 2019
    • October 2019
    • September 2019
    • August 2019
    • July 2019
    • June 2019
    • May 2019
    • April 2019
    • March 2019
    • February 2019
    • January 2019
    • December 2018
    • November 2018
    • October 2018
    • September 2018
    • August 2018
    • July 2018
    • June 2018
    • May 2018
    • April 2018
    • March 2018
    • February 2018
    • January 2018
    • December 2017
    • November 2017
    • October 2017
    • September 2017
    • August 2017
    • July 2017
    • June 2017
    • May 2017
    • April 2017
    • March 2017
    • February 2017
    • January 2017
    Categories
    • News
    Meta
    • Log in
    • Entries feed
    • Comments feed
    • WordPress.org
    Tiatra LLC.

    Tiatra, LLC, based in the Washington, DC metropolitan area, proudly serves federal government agencies, organizations that work with the government and other commercial businesses and organizations. Tiatra specializes in a broad range of information technology (IT) development and management services incorporating solid engineering, attention to client needs, and meeting or exceeding any security parameters required. Our small yet innovative company is structured with a full complement of the necessary technical experts, working with hands-on management, to provide a high level of service and competitive pricing for your systems and engineering requirements.

    Find us on:

    FacebookTwitterLinkedin

    Submitclear

    Tiatra, LLC
    Copyright 2016. All rights reserved.