Examining California’s New AI Chatbot Bill SB 1119

NICOLE LOPEZ: I would like one sentence from each one of you on what you think of the bill. So, Jeremy, you are our state legislative expert… I’d like a sentence from you.

JEREMY KUDON: California just killed a fly with a rocket launcher.

NICOLE LOPEZ: Love it. Chris, you are our appellate and con law whiz. I know that you’ve tracked lots of challenges to California laws that made their way to the Ninth Circuit and beyond. What’s your take on it?

CHRIS CARIELLO: So, as an appellate lawyer, it would never match Jeremy’s concision, or his vivid imagery. But this is a staggeringly ambitious regulation on speech, on access to information, that I think is going to keep the Ninth Circuit very busy, and may well begin to raise some of what I think are the most profound First Amendment questions of our generation. 

So Nicole, what sentence can we get from you?

NICOLE LOPEZ: I’m going to be pithy as compared to you, Chris. No, I’m just teasing. I would say… great intentions. Not a great outcome.

NICOLE LOPEZ: An unnamed voice assistant that takes your orders and only remembers that your weekly coffee grinds need a refresh. Is that in or out?

JEREMY KUDON: That’s easy, that’s out.

NICOLE LOPEZ: Let’s change it up. What if the voice assistant remembers your voice and asks personal questions about your caffeine addiction? Like, “Hey, Jeremy, do you need to consider cutting down on your caffeine intake?” Is that in or out?

JEREMY KUDON: I mean, I feel like this is so personal. I think that that’s… that feels in.

NICOLE LOPEZ: Did I shame you? Did I out you? If you felt something, it’s probably in.

JEREMY KUDON: I feel… I think that’s an in. I think you’ve convinced me that’s it.

NICOLE LOPEZ: So then, what if you tell them that you’re jonesing for coffee, and you had a hard day at work, not because of me, and they start asking if you want to talk about it? Is that in or out?

JEREMY KUDON: I mean, it’s just scary that you actually have access to everything I’m doing… I’m saying and doing in my devices. That’s in. That’s in.

CHRIS CARIELLO: These restrictions… the last one here, excessive praise or flattery disproportionate to the context. I mean, that is something I do every day when one of my kids shows me their artwork.

That is a really subjective concept. What’s flattery? What’s disproportionate? How do you analyze what the user is doing and understand the behavior of the chatbot?

Attempting to diagnose or treat physical, mental, or behavioral health, right? What about acne advice? What about tips for maintaining focus during an exam? You can sort of imagine what the drafters are getting at. It’s probably not those things, but the definition covers it.

And then the last one I would note is discouraging taking breaks. You could imagine a wonderful application of this technology to a purely educational tool. What if that tool tells a kid to stick with it, keep trying, you can get it? What if it imposes time limitations for completing a task, and motivates a kid that way? Is that discouraging a kid from taking a break?

You can imagine the intentions of why you would not want to discourage breaks, but then this immense breadth of the statute — really very little narrow tailoring.

NICOLE LOPEZ: Totally, and that example you just gave is one of the positive use cases of AI tutoring chatbots. So I think this is problematic on so many levels.

JEREMY KUDON: Do you think there’s a chance the states will go farther and ban teen access?

NICOLE LOPEZ: So I think there’s a little bit of a conundrum, right? So, social media bans are a hot topic right now, and they’re clearly divisive.

The fact that we’re seeing schools in LA and also now New York ban chatbots or AI access in classrooms. If we continue to panic in this way about chatbots, I don’t think it’s off the table. But I’m curious, Jeremy, what you think.

JEREMY KUDON: I mean, ultimately, a chatbot is just a user interface to get to the AI, so it’s very different than social media in that sense. And what I can see happening is something very akin to a world I’m very familiar with, which is the online gambling world.

You’re going to end up having a couple dozen companies that are going to follow the law and the rules and do everything required by SB 1119, and then you’re going to have hundreds of others, because when laws are too hard to comply with, you’re just going to have an unregulated market, you know, of these illegal apps.

They’ll all be based in, like, Curacao. And when the AGs go after them, they’ll just shut down, and then they’ll come back with another version the next day. We’ve already seen this play out in gambling, and I fear that we’re now going to see it play out in companion chatbots.

JEREMY KUDON: Over 800 bills were introduced this year alone. 60 of those involved youth companion chatbots. 90 of them overall involved companion chatbots. Why so many companion chatbot bills? It’s not just the social media.

It’s really also, as some might remember, President Trump issued an executive order in December of 2025, where he effectively conditioned, or said, that if a state passes a frontier model developer bill, it could lose its BEAD funding, which for many states is a big amount of money. But it carved out youth — anything involving youth issues — which is why I think you’re seeing so many of these companion chatbot bills this year. It was the only thing the states could do.

But what’s interesting is, of those 60 bills, not a single one required a business to conduct a pre-release risk assessment of their chatbot until SB 1119.

Nothing in California law, or any other state law, requires a developer to assess the risk of, for instance, how it’s training its AI agents, the instructions it gives those agents, or the security features the company has installed to ensure those agents don’t escape into the wild.

California is about to demand more from companies that have companion chatbots than it is, or any other state is, asking frontier model developers.

ROB MCKENNA: We've been tracking 307 separate state AG enforcement activities since 2020 — that's through April of this year, though there's more now, of course, as we sit here in the fall. Those enforcement activities include AI-specific actions in over 11%, almost 12%, of those 307 actions. In other words, AGs are investigating AI in terms of unfair deceptive AI practices, chatbots, consumer-facing AI, and discrimination and algorithmic bias. The AGs have never been more active when it comes to enforcing and investigating around artificial intelligence and its uses.

Notably, the amount of multi-state coordination — that is, actions involving five or more state AGs — is a big part of this. 25% of the enforcement activities we've been monitoring involve five or more state AGs, so 77 actions out of 307. And also notable is that multi-state actions grew from only three in 2020 to 31 multi-states in 2025 — that's a tenfold increase in that time period.

So, we can expect to see these numbers continue to grow and AGs to become more and more active across a range of AI issues. We're here to help if there are any questions about what the AGs are up to when it comes to AI investigations and enforcement activities — just let us know.