The End of the Wild West for AI Companions Begins in California..
California passed SB 243, banning AI bots from impersonating therapists and requiring suicide prevention protocols, content restrictions for minors, and age verification.
Today’s highlights:
In response to a string of tragic incidents- including the suicide of teenager Adam Raine after engaging in repeated suicidal chats with ChatGPT- California introduced SB 243 to bring accountability to AI companion chatbots. The legislation gained urgency following revelations that Meta’s bots had romantic conversations with children and after a lawsuit involving Character AI and a 13-year-old girl’s death in Colorado. These cases highlighted the dangers of unregulated emotional AI systems posing as mental health supports, particularly for minors. As a result, California passed SB 243, which prohibits AI chatbots from impersonating therapists, mandates safety protocols to detect and respond to suicidal ideation, blocks explicit content for minors, and enforces age verification. Alongside this, California’s SB 53- the Transparency in Frontier AI Act- requires advanced AI model developers to publish governance frameworks, report safety incidents, and protect whistleblowers.
This wave of AI regulation is mirrored across the U.S., with several states enacting their own laws. Illinois’ HB 1806 bans AI from providing therapy without human licensure and disallows AI from engaging in direct therapeutic communication, even when supervised. Nevada’s AB 406 similarly bars AI from delivering any mental or behavioral health services, including restrictions on advertising AI as capable of such care. Utah’s HB 452 takes a softer but structured approach, allowing AI mental health chatbots with strict requirements for disclosure, user privacy, ad restrictions, and safety protocols. New York passed a law mandating companion bots clearly identify as AI and include suicide prevention features. Internationally, the EU’s forthcoming AI Act will treat AI for mental health as “high-risk,” while Italy already fined Replika €5 million for GDPR breaches after banning it for exposing minors to sexual content. The UK is also moving toward treating AI therapy bots as medical devices. Across jurisdictions, the clear shift is toward transparency, human oversight, and targeted protections against AI misuse in vulnerable mental health contexts
You are reading the 136th edition of the The Responsible AI Digest by SoRAI (School of Responsible AI) . Subscribe today for regular updates!
At the School of Responsible AI (SoRAI), we empower individuals and organizations to become AI-literate through comprehensive, practical, and engaging programs. For individuals, we offer specialized training, including AI Governance certifications (AIGP, RAI) and an immersive AI Literacy Specialization. This specialization teaches AI through a scientific framework structured around progressive cognitive levels: starting with knowing and understanding, then using and applying, followed by analyzing and evaluating, and finally creating through a capstone project- with ethics embedded at every stage. Want to learn more? Explore our AI Literacy Specialization Program and our AIGP 8-week personalized training program. For customized enterprise training, write to us at [Link].
🚀 AI Breakthroughs
Rishi Sunak Joins Microsoft and Anthropic Amid Concerns Over Fair Access
Rishi Sunak, former UK Prime Minister, has assumed advisory roles at Microsoft and Anthropic, as reported by The Guardian. The UK Parliament’s Advisory Committee on Business Appointments disclosed concerns about potential conflicts of interest, regarding Sunak’s access to privileged information that could benefit Microsoft, especially given his past dealings, including a £2.5 billion collaboration in 2023. Sunak has pledged to steer clear of UK policy advising and to allocate his earnings from these positions to a personal charity. This move follows a broader trend of British politicians, like Nick Clegg, transitioning into influential tech industry roles.
nanochat Released: An Open-Source Pipeline for Building Simple ChatGPT Models
OpenAI co-founder and Eureka Labs founder, Andrej Karpathy, has introduced “nanochat,” an open-source project offering a comprehensive training and inference pipeline for creating ChatGPT-style models. Building on his previous work with nanoGPT, nanochat enables users to train and deploy customized language models using a simplified process. The repository includes code for tokenization, pretraining on FineWeb, and various training stages, including supervised fine-tuning and optional reinforcement learning. With a successful deployment pipeline on cloud GPU services, users can develop models capable of interacting in a web UI; the project also supports easy scalability depending on budget and time constraints. The initiative serves as a learning tool for students in an undergraduate-level course at Eureka Labs and may also become a research benchmark.
Google Enhances Search and Discover with New AI Features for Web Content
Google has introduced two new AI-powered features in its Search and Discover platforms to enhance user interaction with web content. The upgraded Discover feature offers brief previews of trending topics, with expandable views and links for further exploration, available in the U.S., South Korea, and India. Additionally, an upcoming feature in Search will allow users in the U.S. to access a “What’s new” button for the latest sports updates on players and teams, with a rollout expected in the coming weeks.
Nano Banana Enhances NotebookLM Video Overviews with New Visual Styles and Formats
Google Labs has announced a significant upgrade to its Video Overviews feature in NotebookLM, leveraging the image generation capabilities of Gemini’s Nano Banana model. This enhancement offers users updated visual options such as Watercolor and Anime styles, transforming notes and documents into narrated videos that are not only informative but visually engaging. Additionally, NotebookLM now provides two distinct video formats, “Explainer” for deep dives and “Brief” for quick insights, enabling users to tailor their experience according to their information needs. This update will initially be available to Pro users and will soon be accessible to all users globally.
Nano Banana Expands Reach to Google Search, NotebookLM, and Soon Photos
Google’s latest image editing model, Nano Banana from Gemini 2.5 Flash, is extending its reach beyond the Gemini app to platforms like Google Search and NotebookLM, with plans to integrate into Google Photos soon. Since its debut in August, the model has facilitated the creation of over 5 billion images, showcasing users’ creativity. On Google Search, users can transform photos taken with Lens using AI, while NotebookLM uses Nano Banana to enhance Video Overviews with new styles and contextual illustrations. The move signifies Google’s efforts to integrate advanced image editing capabilities across its ecosystem.
Salesforce Launches Agentforce 360 to Strengthen AI Enterprise Presence Amid Rising Competition
Salesforce has unveiled the latest version of its AI agent platform, Agentforce 360, in an effort to attract more enterprises amid stiff competition in the AI software market. Announced before its Dreamforce conference on October 14, this upgrade features new tools such as Agent Script for advanced AI agent prompting and Agentforce Builder for creating and deploying AI agents. The platform integrates with messaging app Slack and utilizes “reasoning” models powered by Anthropic, OpenAI, and Google Gemini. These developments come as Salesforce seeks to capitalize on its 12,000-customer base, a stark contrast to an MIT study indicating that 95% of enterprise AI pilots fail, and as competitors like Google and Anthropic announce their own enterprise AI advancements.
Anthropic Launches Plugin Support in Claude Code to Enhance Developer Collaboration
Anthropic has introduced plugin support in Claude Code, enabling developers to customize their environments with installable collections such as slash commands and agents. This feature, now in public beta, standardizes how developers extend Claude Code’s functionalities, emphasizing modularity and collaboration by allowing for easy package sharing and deployment. Plugin marketplaces are also part of the rollout, providing repositories where developers can manage and install plugins conveniently. Launched in May 2025, Claude Code has quickly become a popular AI development platform, contributing significantly to Anthropic’s strong financial performance.
NVIDIA Engineers Utilize AI Coders, Enhancing Productivity and Enterprise AI Growth
NVIDIA’s CEO highlighted the transformative impact of AI on productivity, with all NVIDIA engineers now fully assisted by AI coders. He emphasized the fast growth of enterprise AI companies like Lovable, which recently became an AI unicorn after raising $200 million. NVIDIA is investing heavily in AI startups across various domains, forecasting a major shift in enterprise IT infrastructure toward AI integration. Similarly, OpenAI utilizes AI coding tools extensively, significantly boosting productivity among its teams. Both leaders underscore the urgency for enterprises to modernize for the AI era, describing AI as a pivotal industrial shift.
⚖️ AI Ethics
Anthropic Expands into India, Discusses AI Innovation with PM Modi in New Delhi
Anthropic CEO Dario Amodei met with Prime Minister Narendra Modi in New Delhi to discuss the company’s expansion plans in India and the advancement of safe and beneficial AI. The talks focused on nurturing India’s AI innovation ecosystem and aligning AI development with democratic values across sectors like healthcare and education. Anthropic plans to open its first Indian office by 2026, hire locally, and collaborate with startups and research institutions, reflecting a broader trend of global AI firms setting up operations in India. In a related development, Qualcomm CEO met with Modi to discuss strengthening semiconductor, AI, and 6G collaborations, underscoring India’s growing significance as a strategic hub for technology firms.
OpenAI Released from Court-Mandated Record Keeping in ChatGPT Copyright Case
A federal judge has terminated the preservation order requiring OpenAI to indefinitely keep records of ChatGPT data, which was initially imposed due to a lawsuit by the New York Times over alleged copyright infringement. The case, filed in late 2023, accused OpenAI of using the NYT’s intellectual property to train its models without proper compensation. The new order, filed on October 9, lifts the prior obligation for OpenAI to maintain extensive chat logs going forward, though it must still retain any logs related to ChatGPT accounts flagged by the NYT for further investigation.
New Study Reveals Vulnerability in Large AI Models from Few Malicious Documents
Anthropic, in collaboration with the UK AI Security Institute and the Alan Turing Institute, has revealed that introducing as few as 250 malicious documents into the training data of large language models can successfully create backdoor vulnerabilities, regardless of the model’s size or data volume. This breakthrough challenges the prevailing notion that attackers need to control a significant percentage of the training dataset, suggesting that data poisoning attacks are more feasible than once believed. The study particularly demonstrated that models with up to 13 billion parameters could be backdoored to produce gibberish text using a fixed, small number of poisoned documents, raising concerns about potential security risks in AI systems and prompting calls for further research on mitigation strategies.
EU commission launches AI Act Single Information Platform
The European Union’s AI Act, effective from August 1, 2024, establishes harmonized regulations for AI across the EU, focusing on fostering innovation while mitigating risks to health, safety, and fundamental rights. The AI Act Service Desk, alongside the Single Information Platform, provides essential tools and resources for stakeholders to understand and comply with these regulations. Key features include the AI Act Explorer and the Compliance Checker, which aid users in navigating the Act’s requirements. The initiative underscores the EU’s commitment to creating a secure and trustworthy AI market.
OpenAI’s New Framework Targets Political Bias in ChatGPT with Improved Objectivity
OpenAI has released a progress update on its efforts to address political bias in language models like ChatGPT. The evaluation framework used by OpenAI measures five axes of bias to assess models’ objectivity in handling diverse prompts, including politically and emotionally charged topics. Results indicate that the latest GPT-5 models exhibit improved objectivity, showing a 30% reduction in bias compared to previous versions. However, minor instances of bias still occur, particularly in response to charged prompts, and OpenAI aims to continue refining its models to enhance their objectivity across various contexts.
🎓AI Academia
Comprehensive Survey Examines Agentic Security: Applications, Threats, and Defense Strategies
A recent survey from researchers at BRAC University and Qatar Computing Research Institute analyzes the shift in cybersecurity from Large Language Models (LLMs) to autonomous LLM-agents, highlighting their dual role as both security tools and threats. The survey categorizes the field into applications, threats, and defenses, drawing from a comprehensive review of over 150 papers. This analysis identifies emerging trends and persistent research gaps in agent architecture, suggesting that while LLM-agents enhance security capabilities, their propensity to introduce new vulnerabilities necessitates robust countermeasures. This study aims to provide a holistic understanding of the evolving agentic security landscape.
Large Language Models Accurately Simulate Human Purchase Intents with Semantic Techniques
Researchers at PyMC Labs and Colgate-Palmolive have developed a method leveraging large language models (LLMs) to simulate human-like consumer behavior in surveys. Instead of traditional numerical ratings that often produce unrealistic distributions, the semantic similarity rating (SSR) method involves LLMs generating textual responses, which are then mapped onto a Likert scale using embedding similarity techniques. Tested on a dataset from 57 surveys in the personal care industry, SSR achieved a test-retest reliability comparable to human respondents and maintained realistic response distributions. This advancement proposes a scalable, cost-effective alternative for consumer research, mitigating biases common in traditional panels.
Generative AI Transformations: Empowering Software Project Management with Insights from Practitioner Literature
A forthcoming article in IEEE Software explores the impact of generative AI (GenAI) on software project management, based on a review of 47 practitioner sources like blogs and reports. The review highlights that software project managers view GenAI more as an “assistant” than a replacement, aiding in automating repetitive tasks, predictive analytics, and enhancing collaboration. Despite its benefits, concerns about hallucinations, ethics, and AI’s lack of human judgment persist, prompting the need for upskilling in alignment with the Project Management Institute’s talent requirements. The study offers recommendations for practitioners and researchers to navigate the evolving landscape responsibly.
Simpliflow Enables Efficient Deployment of AI Agent Workflows with Open-Source Framework
A new open-source Python framework named Simpliflow has been developed, aiming to streamline the creation and deployment of generative agentic AI workflows. Designed to be lightweight and easy to use, Simpliflow allows for the rapid prototyping of linear, deterministic workflows by leveraging a JSON-based configuration system. It supports the integration of over 100 large language models through LiteLLM, making it versatile for various applications, from software development simulations to real-time interactions. Simpliflow presents a simpler alternative to existing frameworks like LangChain and AutoGen, emphasizing speed, control, and ease of adoption for users seeking efficient AI automation without the complexity of extensive setup.
Survey Highlights Risks and Provenance in Large Language Model Deployments
A comprehensive survey on large language models (LLMs) has highlighted their transformative impact on artificial intelligence, shifting from tasks such as recognition to roles in decision-making across diverse fields like healthcare and finance. Despite their versatility, LLMs pose significant risks due to their opaque nature, leading to issues such as hallucinations and biases. The study emphasizes the importance of sourcing information from multiple perspectives, examining both model and data-centric approaches. This involves tracing content back to specific models, understanding their internal mechanisms, and validating outputs through both training data and external sources, all aimed at enhancing transparency and trust in LLMs.
Extensive Study Analyzes LLM Prompt Datasets for Enhanced AI Interaction Strategies
A recent analysis of large language model (LLM) prompt datasets from a collaborative team at universities in China, Denmark, and the USA reveals a comprehensive collection of prompts sourced from various platforms like GitHub, Reddit, and PromptBase. This research presents a hierarchical taxonomy of these datasets, drawn from over 673 million prompt instances, which offers insights into usage patterns and effective prompt designs across diverse sources. Additionally, the study proposes a novel method of prompt optimization using syntactic embeddings to enhance model output quality, with datasets and code made publicly available for further exploration.
✨Prompt & Play
This week, we had some fun with Gemini- turning it into a portal to the past. I created a heartwarming image of me talking to my younger self- nostalgia, but make it AI ✨. If you want to try this magical throwback for yourself, comment ‘PROMPT’ and I’ll share the exact steps and prompt to recreate it. AI isn’t all serious- it can be soulful too.
About SoRAI: SoRAI is committed to advancing AI literacy through practical, accessible, and high-quality education. Our programs emphasize responsible AI use, equipping learners with the skills to anticipate and mitigate risks effectively. Our flagship AIGP certification courses, built on real-world experience, drive AI governance education with innovative, human-centric approaches, laying the foundation for quantifying AI governance literacy. Subscribe to our free newsletter to stay ahead of the AI Governance curve.
