DEF CON Meets MCSC 2026: Why policy hast failed to address the risks of next gen technologies?

Okay.

So for this panel, we've heard a little bit about the, the mm- movement of AI, we've heard about civil society and, um, maybe some agency around that.

In this one, we're gonna talk a little bit more maybe about the role government regulation, um, some of the things we can possibly do about this, why haven't we done some of the, what we would consider easy things by now.

Uh, and so, uh, Daniel's involved with, uh, UK Government Cyber Advisory Board.

Jake was at ONCD.

Uh, Luis was, uh, i- in a prior life at ICANN, so a bit of internet...

yeah, a little bit of internet governance.

Now, you're at RUSI, so a little bit of think tank, right?

Perri was a special assistant to the director of DARPA, so a little bit of insight on how research agencies and governments think about this.

So we've got a mix of, of perspectives.

Um, one of the things I, I want people to think about, and I realized I didn't bring this up earlier, um, was why, why do we think there's only one Internet Archive and it's in America?

Why is there one Signal in America?

Why is there one Wikipedia in America?

Why is there one Let's Encrypt certificate authority that provides free certificates for the planet in America?

Come on, guys.

Get with the fucking program.

We need some more diversity.

Th- there's a fundamental problem.

Why aren't there three in India and five in China and eight in Europe?

Like, something is going on here and we don't understand it.

If Signal's $50 million a year, let's say Let's Encrypt's $50 million a year, I think I saw some math that Internet Archive could do about $100 million a year.

Well, we're talking $400 million, $500 million, and nobody is willing to put that up, but we'll spend billions on mining, you know, Bitcoin or something.

So I just wanna...

I wanna say that there's, there's something going on where either governments aren't prevening- providing the correct incentive structures, or for some weird reason, all the incentives exist in the States and they don't exist anywhere else, which I don't, I don't believe.

Um, okay.

So, the topic for this panel is regulate the outcome, um, and we talked a bit about it briefly before.

W- we seem or government seem obsessed with regulating the steps, not the outcome.

And my goal here with bringing this up is if we could regulate...

if we could think about the outcome, ch-...

let's say, as a society, we decide, uh, privacy-preserving technology should be the norm, or, uh, there should be low tr- uh, switching costs.

I'm on a social media platform, I should have the right to back up my social media identity and be able to transport it somewhere else.

It should be in a machine-readable format.

Um, we should have the right to repair.

Whatever the foundational model we go with then can be applied to whatever the generation of technology is.

So, if it's a right to repair, what does that mean to AI?

If we have the right to be transportable, what does that mean in AI or in any other model?

But since we don't seem to ever really regulate the outcome, it doesn't feel like we'll ever get ahead of these situations.

So, in, in, in various work and when you're advising people, um, am I fixating on the wrong thing?

Is regulate the outcome a worthwhile goal, um, to increase our, uh, government's ability to regulate intelligently, or are we just gonna be talking about...

And I, I...

one other aside Jake and I were talking about.

We've been going to security conferences for 20-plus, 30 years, and, you know, we have not said we need, uh, threat intelligence-sharing, we don't need...

what was the other one?

Uh...

Public-private.

...

public-private partnership, right?

You're gonna get kicked off the stage if you were saying that.

There's, uh, e-...

there's just certain things that we always fall back on as crutches that never deliver, but yet we keep repeating them, and so I'm trying to move the conversation away fr-...

Workforce development, yeah.

Okay.

Go for it, guys.

Perri's in for the kill.

Things that Perri's going.

Okay.

Yes.

Oh, dear.

I, uh...

I put myself on the chopping block with that one.

Yeah.

Uh, as you're talking, I, I...

and I am going to give the caveat that I am coming on stage after Meredith and I'm about to say something about cryptography.

Uh, and it will only be a letdown.

But, uh, eh, something that strikes me as you're talking about a number of these technologies and government, uh, intervention and regulation is the export control space, which had really deleterious effects on, uh,

cryptography and the ability to make and sell cryptographically secure products in the, uh, 2000s in the United States.

This is also true more broadly.

And and that was a space where, in some ways, there were lessons learned.

These, uh, controls were relaxed, but now we're starting to see the same approach occur with AI and regulating how that is shared, regulating how that is used, and this is coming at a time where there's a number of open source models.

And so, as you're talking about cert-...

uh, uh, certs, free certs, as you're talking about the Internet Archive and Wikipedia, right, one also, uh, thinks that the next generation of that is going to be these open source models, these frontier models, and how those are...

how those are regulated and who is allowed to put those out and who is allowed to use them and package them.

And I think there is a significant risk to trying to, as you put it, regulate the, uh, uh, regulate the steps.

...

rather than regulate the actual outcome and how it is used.

Because you're going to really, uh, uh, uh, dampen creativity, you're going to dampen the ability to create, and you're going to s- uh, centralize it in the hands of, uh, a few, uh, companies, governments, you know?

A- and it reminds me of, like, in China th- their approach to AI is they do regulate the outcome of AI.

It has to be, uh, a politically accurate result from the search.

And you can argue all day long that that's not technically possible, these models will spit out other things.

And the political apparatus in China has decided that's irrelevant, right?

Their- the outcome must be conformant.

Uh, so you don't necessarily even have to regulate reality.

A- although, I mean, w- we're no better, uh, at least, uh, you know, across, uh, the- the West.

I know when...

in the Biden administration, we were, um, you know, very worried about AI being racist and sexist and all these things, and we were, like, really focused on the executive order and so on, trying to say, "Well, you should only train your data on...

or you...

to only train your models on this data that's gonna- gonna stop it from being racist and stop it from, you know, having bias and so on."

Um, and then, of course, Musk and those guys come in and say, "Oh, you're woke GPT," you know?

It shows all the- the famous picture of the people signing the De- the Declaration of Independence, and they're- they're not all white men, which, of course, is who signed the Declaration of Independence.

Um, and so, like, we're...

uh, I- I don't know how much bad...

I mean, okay, fine, we're not as bad as the Chinese.

But we're...

I don't know if we're much further down the road in a good way, uh, than they are.

I think, I think it's usually...

I mean, to your point, Geoff, why is it...

like, why are we not regulating outcomes?

To be honest, because it's easier not to, right?

It requires thinking about what Meredith had mentioned about the structural incentives, and maybe we are not up for conte- you know, contesting some of those.

And it's interesting, as you were talking, I was thinking about something that's potentially quite boring nowadays, because now everything's about AI, but I was thinking about the discussion about the right to be forgotten, you know, good old days, where we were talking about GDPR and, like, the

implementation of GDPR, right?

And I remember these conversations, these fiery conversations, you know?

Back then, I was working in, like, civil society academia, and we were talking about, like, how is Google gonna de-index, like, all of these kind of things, right?

And that, for me, is not regulating outcomes.

That is, again, another example of how we focus on these intermediary steps instead of going to the structural incentives, right?

Um, and that, and that's why I think, you know, again, to your point, why don't we do that?

Because it's easier.

Are we really, really willing to...

on the AI conversation?

Because nowadays, the right to be forgotten has become obsolete, you know, with- with the context of AI, with how much data can be scraped and how much you can index if you ask AI, "Give me some information on this name."

Or even Google, right?

I mean, with Google's, like, AI feature as you search for a person's name, like, it will show up, like, lots of things and a summary that's pretty accurate of that person, right?

Um, so that, for me, is a clear example of our- our- our lack of clarity.

I don't think it's lack of, um, willingness.

I think it's lack of clarity of how we go from point A to point B, which point B being regulating the outcomes.

I- I want to add to that.

I think it's easy, but I also think there's a political element.

It's also scoring political points for the voting masses, right?

So, let's take Australia with their wild thing of banning social media for people under 16.

Job's good done, right?

We've done it.

We fixed the problem.

W- it's too hard to go after tech giants, 'cause they're- they're big and tough.

So, what we've done is we've made life miserable for kids under 16, right?

Voters love it, so you don't have to use social media.

Unfortunately, you're gonna have the situation where kids are going behind the bike shed to do social media, like they do cigarettes and alcohol and everything else.

But it scores political points because the voters go, "Look, the government's doing something."

It's regulating the tools out of existence rather than doing the harder thing of, actually, what are we trying to solve?

I was gonna...

Uh, we were in the green room a little bit earlier, and we were joking around, like, "Well, okay, what- what is the regulated outcome model that we could hold up as a success?"

And I think, I think we had, um...

It's illegal to prevent you from unsubscribing to a mailing list.

Like, it's a spam solution.

There's regulation pretty much everywhere that says it's criminal to not allow somebody to unsubscribe from an email.

That's about it.

We regulated, like, one spam outcome.

And how successful is that?

Do we still get spam, you know?

So, I kind of, I'm kind of with you that- that- the incentives must be wrong or something's misaligned, but it seems like if, uh, regulating the steps isn't working, what are we left with?

No regulation or regulate the outcomes?

Maybe it's time to try the outcomes part, and maybe the outcomes requires more involvement from civil society or researchers or academics in the technical area, because we can speak truth to power.

We can say, "Hey, this is realistic.

This is not realistic."

Because I don't...

Microsoft might not tell you this is the- the truth, right?

So you need a third party's sort of validation, I guess.

Can I push back on something you said-

Yes.

...

at the beginning?

I hear this is the most...

uh, the best way to do panels is to get a little contentious.

Exactly.

Spicy.

Um, so I think that the flip of having centralization, uh, and singularity in something like Signal and something like Wikipedia is those white blood cells that Meredith was talking about, is that because Signal is the app, you have a

lot of eyes on it from a security point of view.

Some of the best researchers are looking at it.

You have an all of community response to trying to secure it.

Wikipedia, right, is the same.

The only reason Wikipedia is what it is today is because of all of the volunteers that contribute to it and keep it, uh, uh, as the internet's encyclopedia.

And one has to ask, if you start to really Balkanize that, are you going to have the same result?

Is there a value in having that centrality?

Or is it more distributed, where the German Wikipedia community has the German Wikipedia?

They- they edit the hell out of that, and it gets pushed, distributed to all the other distribu- distributed Wikipedia instances everywhere.

And so...

You might be able to, I don't know, engineer a sort of takeover of part of Wikipedia, but it's a federated connection of Wikipedias, not a central.

You want a Wikipedia blockchain.

Uh, blo- uh, I was thinking more Web3- ...

with an NFT.

Do we, do we kick him off the stage for saying blockchain?

For having jokes.

Don't worry about it, yeah.

Okay, so then my question is how, i- if you are gonna try to regulate outcomes, how do you get there?

Yeah, I mean, I, I think that-

And I don't wanna fixate on that one, Tom.

I don't wanna make this about regulate the outcomes, but I think...

I kind of wanna understand, how do we not have the same conversation five years from now?

Yeah, well, I mean, in cyber, we're always having the same conversation.

But, uh, I, I...

What I think, and if you, if you go back to the example of, you know, the Biden administration don't, don't train your models on the, on data that, you know, could make your models racist and so on.

And then Musk's point of, you know, woke GPT blah, blah, blah is a bad thing.

I think about my, my kids who will, like, look at a video and be like, "Oh dad, that's totally fake.

That's clearly AI generated.

What are you talking about?"

Like, and I think that if we can figure out a way to regulate an outcome where we get trust or we get the ability to know when things are not real, so then people can decide, okay, I do

want...

Like, if I want my kids to be scrolling TikTok eight hours a day at the ripe old age of seven, like, okay, that's your r- you know, that's your decision as a parent, fine.

But, uh, the world that I think we're, we're moving into, um, is, is one in which it's nearly impossible to decide what's actually real, um, and what's not.

And I think that if we could get to the bottom of figuring out that, then the other thing's kind of-

So your truth, truth in advertising-

Yeah, tha-

...

transparency in...

Or just the ability to know when it's fake.

Then I think we can kind of start to sift through the rest of the shit, you know?

I, I agree.

So, so I've got twin boys who are nine, they play Roblox and speak a language they don't understand anymore.

Um, they love YouTube Shorts.

As much as I dislike that brain rot, um, I want the ability not to block them from seeing it, but I want to curb how much time they have.

And then the same thing with, with AI and deepfakes and generated content.

I want YouTube to do more.

I don't believe it's a technical hurdle they can't fix.

I want it so that I can have a flag in the settings that says, "My kids will not see deepfake or AI-generated content."

Right, and that's something that we could regulate the big, uh, tech providers to say, "You, you must."

Like, as if they don't know.

Of course they know, and of course they can do this.

Um, uh, that regulation is completely possible.

And that's an outcome thing, right?

It's not blocking YouTube Shorts.

It's just saying, "You will ensure that your models stop that."

Just maybe stepping back and asking the question of what is the outcome that we want?

And do we have consensus to do that?

Uh, because as we're talking about these measures, right, I- I'm, I'm thinking of, like, the election cycle, like the big elections year, right?

I mean, many, like, companies such as Facebook, well, back then Facebook Meta, um, created, like, war rooms and they created, like, they flagged content that was linked to certain parties or certain, you know, that was, you know, moneti- you know,

paid or adverts from, let's say, certain parties.

That's what they could do.

That's what they did.

Again, that's a piecemeal approach and that's also kind of insufficient when you think that right after that cycle, like, most of the big platforms did cut the funding for their trust and safety teams.

So that has something to say about do we have the same outcomes that we're trying to reach?

Um, and, and I guess that's an- a question that we need to ask.

Uh, uh, now on the AI side, like, in November last year, um, I mean, now thinking about what governments can do and maybe take us to the, you know, conversation to, like, co-regulation.

Um, last year, you know, um, the, the, the European Commission had to, like, put on a digital omnibus and then it's, it basically admitted that it had to stop certain types of implementation or had to delay the implementation of certain

parts of the EU AI Act because they didn't want to stop innovation, right?

And I think that, that shows that there is this uncertainty or at least this, this need to balance, at least on the European side of things, this, this, you know, impetus or let's say the European soft power to

regulate and to be a regulatory power with, at the same time, trying to bring innovation.

That is, again, pushing the conversation.

That is what the EU is trying to balance in terms of which outcomes it's trying to reach, right?

And it, it didn't necessarily get to an answer because when we look at the other side of the coin so that we can take the conversation hopefully to, like, co-regulation, right?

If the end goal is to ensure that we all have a safe or, or secure kind of, um, use of LLMs and these models, right, how do we make sure that it's not just regulation trying to kind of speak externally to that, but that,

you know, we can have more understanding of how these models operate?

And actually, like on the work that we've been doing, and that Dan has also been kind of like doing, when looking at how third parties access Frontier AI models, you know, what are the risks of third parties accessing?

How can we make sure that that is secure?

How do we make sure that there is external validation of some...

Not validation, but let's say at least some oversight or potentially kind of some form of assurance of how these models operate, if they operate with security in mind, if developers could consider something extra?

Because obviously these companies are overstretched when they think about securing their own development of LLMs, right?

I mean, their teams are limited.

So can we think about new ways or new avenues of collaboration that can, you know, not just provide companies what they need, but also with, let's say, external party evaluators or red teamers, but also on the regulatory side,

how, what is the kind of information that we need to provide in order for regulation to be effective, right?

So, so that's the- How does civil society help provide the information to provide better policy outcomes, or...?

I think it's, it's...

We need to look at this, from my perspective at least, we need to map the different stakeholders and which place they occupy in this conversation.

If we're talking about security and safety of, of frontier AI models, I think there needs to be greater integration between communities.

What, what we found is that, you know, obviously evaluators are the third party-

Sure.

...

assurers of many of these, uh, frontier models.

But sometimes the cybersecurity community isn't as integrated, right?

And are we talking to, like, you know, uh, I mean, obviously this is in the context of assuring that a certain model is ready to be deployed, and even in pre-deployment, right?

Anybody?

Mueller?

Uh-

Go ahead.

I've...

Okay.

I-

Because I don't want it to be all about AI, but it is a good example of-

Uh, this, this will be a little about...

Yeah.

...

about AI.

But, you know, something that strikes me about what is the, what is the intended outcome, uh, especially with, with, with Signal, with Wikipedia, with, uh, frontier models.

Uh, yes, there is, uh, government influence and pressure and, in some cases, regulation to achieve certain outcomes.

But the end result, the product often reflects the values of that company and of that entity.

And this is not a, uh, uh, this is not me saying public-private partnership, but it's simply me saying that, uh, uh, there is a, a soft approach here as well, which is aligning the, uh, folks who are on, on the

forefront and who are achieving, uh, uh, cus- uh, largest customer base are achieving community interest.

Uh, uh, have those organizations be the ones that reflect the values that you're, uh, uh, that you would like to see in these end products, that is going to achieve the best result here.

There's, um...

I- I'm, I'm gonna go there.

Um, who uses Mastodon?

Anybody?

Or heard of the Fediverse?

Okay.

So there's this...

I- if you remember back years ago, the FBI got their hands on an iPhone that was used in a, a terror suspect in the United States.

And the FBI was trying to force Apple to unlock the phone.

Apple was resistant, they didn't want to unlock the phone.

Um, and it went back and forth.

And eventually, I think Cellebrite unlocked the phone for the FBI.

But after that, what happened is Apple decided that they didn't want to spend all their time with their, paying their lawyers to fight the FBI or any other government in the world.

So they went about engineering themselves out of the problem.

They engineered end-to-end encryption.

The end user has the key.

Apple has nothing.

They just...

They're not gonna spend their budget having this fight.

So they engineered themselves out of the equation, right?

So at some point, you make the regulation or you make something too s- difficult, companies will engineer themselves out of it.

But on the other hand, with the Fediverse, with Mastodon or whatever, if you're not getting the good moderation tools you need out of Twitter, you're not finding the community you want at Twitter, the government's not regulating the white supremacy on Twitter or the

pedophilia autogenerated images, if you're not happy with that, where do you go?

And it turns out in this Fediverse, a giant distributed network, they, the community has expressed their norms, right?

You can find different platforms, you can align with whatever your communities are.

You have strong moderation tools that are not available to you on other commercial platforms, because strong moderation tools makes you less valuable to them, right?

So it almost feels like we're starting just at the beginning point of, if we're not getting what we need from the companies and the government's not delivering for us, civil society is starting to build their own alternative replacement for Instagram,

YouTube, TikTok, X, whatever.

All the replacements are starting to come.

And they're coming in a distributed fashion with strong end user goals, with no algorithms forcing choice down your throat.

And it's, uh, just what you were talking, that triggered this memory that, oh, it's maybe the outcomes are coming now from the software creators.

And they'll only accelerate with automatically better generated source code from some of these AI tools.

Now you could actually build replacements that better reflect the values of the communities, where before you needed 5,000 engineers.

Now you maybe need 5,000 hours.

End rant.

Okay.

Yeah.

Yeah, I think that's accurate.

Uh, I think it's a really interesting question to say, how do LLMs, how does code generation, uh, help enable the open source community to become so much more powerful?

Especially at a time where there is...

while there's so much funding for open source, there's also so many well-paid jobs for developers.

And, uh, uh, it is, I, I think a community in a certain amount of flux right now.

And this is a really excellent opportunity to enable folks that don't have that much time, uh, but want to contribute, want to build things, want to put it out there to, uh, to really do that.

Yeah.

A- and I think, for example, uh, king of what Perry said, um, one of the outcomes we could, we could try and regulate toward is open source for critical infrastructure.

So for example, um, the precursor to the Almanac was the Voting Machine Hacking Village report that we used to put out, um, where, you know, we used...

At DEF CON, we would hack voting machines and voting software and so on, and produce a report on the, the vulnerabilities we found.

And one of our, um...

Uh, and by the way, which is the first time that was ever publicly done.

There had been a handful of assessments of voting equipment before, but the first time it was ever publicly done, um, was then.

...

a, as a quick aside, 'cause we've got 15 minutes.

Um, you know, of course the voting industry and the election officials were all like, "Our stuff's totally secure.

You can't hack this."

Which of course the hackers all laugh at.

Um, and, uh, our first finding was when, uh, one of the guys who was, who was running the village with us, um, opens up the back of one of the machines, and on nearly every single part it said made in,

where?

China.

China.

So, we're like, "These things could be rolling off the assembly line pre-hacked.

Like, what do you mean it's secure?"

And so since then we've been saying, "Why isn't voting equipment or voting software open source?"

Like, it's crazy that it's not open source.

We should have every hacker on the planet looking at, uh, voting software just like they look at, at Signal.

Um, it's a hell of a lot more important than Signal.

No offense to Signal.

Um, but yet we still don't have that.

And we, we could have that for multiple aspects of critical infrastructure.

Uh, and so anyway, that's certainly one of the outcomes we could, we could definitely regulate.

I'm really glad you didn't mention the grain of rice when you did the back door there.

Thank you.

I remember I was there in the room that first year when they, uh, I was standing next to an election official, and he was like, so excited.

And I'm like, "What are you doing here?

Don't you have these machines?"

And he's like, "Oh yeah, we have, you know, 10,000 of these machines."

Or whatever.

But my contract says I'm prohibited from opening up and looking inside.

So I've- ...

I've never been allowed to look inside one of these machines.

And, and I've always been curious.

Um, like, okay, so the guy takes out the screws and this thing, pries it open, that guy's eyes are really big and he's like, "Wow, look at that.

That interface is not in my technical diagram.

That memory controller is not in my diagram."

You know, and he was just like horrified, like everything that he'd been told by the manufacturer and the schematics he has in his hands, and he's trying to line it up and it's like, nope.

It's all different.

And it was like that poor guy, just his whole world crumbled, you know?

Right.

And that was just some screws in the back of a voting machine.

And like you, that's why I'm thinking that, and the only reason that happened was because in America, the Digital Millennium Copyright Act enabled a carve out for good faith research in election security.

And it had just happened.

If that carve out had not occurred, none of the security research could have happened.

And so there are so, right, regulate the outcome.

They wanted to encourage researchers to go after those medical devices, uh, or life safety medical devices, I think, life critical, elections and so forth.

And they had a very positive outcome.

Um...

Maybe that's a, a, a segue into you regulate the outcome where CNI based stuff should be open source.

I know there'll be some distractors, but you know, st- if you look at say, uh, PLC controllers, which history, they've been terrible.

Sorry if anybody works for any of the companies here, they, they have not been good.

But they're closed source and people can't get access to them.

And it's, it's hard that way.

I think one of the powers I see today with a lot of the, the AI models is that you've almost homegrown a whole lot of vulnerability res- researchers that have always been afraid to do it 'cause of gatekeeping.

But now I'm really interested in a PLC that controls water.

Okay, I can see the code repo on GitHub.

I'm gonna learn how it works.

I found a vulnerability to do a whole pool request.

Maybe that's what one of the regulated outcomes needs to be.

It's, if you have something that controls water for millions of people, the code should be accessible for people to view.

I know that's a very typical hacker view to it, but...

So, uh, going back to the, uh, uh, topic that you started with, which is what happens when you try to regulate those intermediate steps.

It reminded me of a discussion I had when I was in government about how to best regulate AI.

And this is at the very beginning of kind of the concern with regard to, uh, AI and models and, uh, so many folks were very reasonably and thoughtfully trying to ensure that we had the best outcomes from that technology.

But what resulted, uh, were some policy proposals that were quite, quite serious, uh, that would say things like, um, and the one I'm thinking of is, okay, well if it generates its own code, if it's code that generates its own code,

then that's something that we need to consider to be higher risk.

Right?

And I'm reading this and I'm thinking, well, every JIT, every just in time, uh, uh, compiler in a browser, which says, oh, you are doing this action over and over and over again.

I'm going to save the machine code so that it just runs faster.

That's generating code.

This is something that's very common and there's so many other examples of that.

And this is a good faith effort to say, Hey, we wanna have a little bit more insight with people are going to regu- uh, uh, generate malware, which of course we should obviously be very concerned if you have people using AI to generate malware that they're targeting

schools and hospitals, et cetera with, right?

But that's the end result.

And if you start to, uh, uh, tailor these fine-grained regulations to intermediate steps, uh, uh, unless they are very, very technically sound, they have these really huge rippling, uh, side

effects that are, are really counterintuitive.

Okay.

I wanna go for the audience.

Wanna go for the audience?

Sure.

Go for audience.

This gentleman first.

I feel like it's gonna be a good question 'cause he was very excited.

He's got a beard.

Right.

Yeah.

Hello, I'm Dean Wiesbarth from Innobyte.

Um, you talked a lot about, of regulating the outcome and that there's not a lot of laws doing that.

I was wondering, do you consider the NIS2 directive to be such a law that regulates outcome?

I have no idea what the NIS2 law is.

I have no idea.

What is it?

Uh, German directive, uh, European directive.

Thank you.

Uh, put into law in Germany last year.

That's why I was ...

if the German.

Um, it basically tells companies of a certain size and certain criticality to be, um, IT secure.

And it doesn't specify exactly how, it just mentions that you have to...

...

do some sort of two-factor authentication.

It has t- uh, to do with, um-

It's like a mature- It's like a maturity model.

You have to try to be this tall, and then each year they'll try to improve it or iterate through.

Uh, they basically give the local regulators, for example, the BSI, uh, the authority to mandate what you should do.

Yeah.

I see this model in China, where they keep increasing the protections on PIPL, their individual personal privacy laws, where, uh, in the very early days, it was your company, if you have a website, commerce site, they would

scan your website, and if they, their tools didn't show any vulnerabilities, you passed, you could stay in business.

Um, and then it got to the point where now they're doing more in-depth, like SQL injections and, and human machine audits.

And now, it's, it's on its third generation.

It's getting pretty sophisticated.

They partner with universities.

Um, the university students use this as part of their security training course where to, like, test, uh, companies.

And it's all part about, uh, China has this vision that the future of China is online, and citizens should trust the online infrastructure.

So, they spend a lot of time doing these kinds of exercises so people will trust...

right?

That, so their outcome was, we need people to trust the internet.

How do we do that?

And they have this multi-year plan, and they just keep ratcheting it up and ratcheting it up.

Uh, I'm familiar with that.

My biggest concern and criticism is that it reminds me of the OWASP Top 10, and I apologize for the OWASP Top 10.

Um, it was never meant to be what people use it as.

And unfortunately, now people say, "Well, we're safe from OWASP Top 10."

I'm like, "There's one of the...

I don't even know what that means."

So, I think the intent's there, it just doesn't go far enough.

I, I actually prefer that-

So, you really, you really believe in the OWASP Top 20 then?

I think, I think there's an AI Top 10 now.

I'm sorry.

Anybody else?

Nope.

Okay.

Uh...

Hi there.

I'm Susanna Diemel from, uh, Bitkom, German Digital Association, and I was wondering, uh, what, what is it exactly what we mean by when we talk about output-related, um, regulation?

Because, um, usually when we think about legislation, we do abstract rules for a unspecified number of cases in the, in, well, we, with regards to technologies, we

sometimes try to regulate the risks of, um, developing technologies in advance, at least in Europe that's, that's quite popular.

But the difficulty is as described.

We, we don't really know what will be the outcomes of the use of this technology.

We don't know how the technology develops.

We don't know, um, what will be the side effects, um, of...

so, so I think it's really hard to, um, to regulate the output and also with in-...

regards to enforcement, you know, you have...

yeah, so, so that's what, what I was wondering about.

What do you th-...

what do you think about that?

You've touched on a really interesting point, that the gap between the pace of technology and the fact that regulators and government are like slow-moving mammoths.

And I think the reality is, let's take the, the YouTube example.

Like, there are plenty of studies out there that now show that the excessive doomscrolling is not good, even for adults, right?

So you almost feel like the tech companies are not doing enough to self-regulate, so you look at mom and dad, right?

And mom and dad are outside having a cigarette.

For me, that's the analogy of government.

Why are they not moving faster?

Why does it take years and years and years to regulate when tech can move in days, weeks and months, and even more so now?

I think that's probably where we need to see the biggest shift, is that government, h- hate to say the word, they, they've got to be a little bit more agile.

I also think you've touched on an uncomfortable truth when it comes to technological regulation, which is that we almost treat society as, uh, uh, human subjects for research.

And we are prepared to let technology go forth in the world and then to assess its effect, and over time, collectively, uh, start to try to constrain it after the fact, which obviously has significant

risks, right?

Uh, but there's a, uh, uh, argument for doing it that way, which is that if you are too risk-averse, you can't see what technology is-

Yeah.

...

actually gonna flourish-

Okay.

But-

...

and improve.

...

if, if you had a strong child protection framework, and a child protection framework might say, um, uh, a- attention distortive, you know, whether it's radio or TV or rock and roll or whatever it is you've decided, video

games in the arcade or something, you decide that these things, uh, chi-...

children under six, like, disturb their attention span and hurt their cognitive capabilities, then fine, the internet comes along, doomscrolling falls right underneath the c-...

the, the child protection stuff for various age-appropriate content, or you...

there's a, there's studies about how many times an image changes on the screen if, if you have fast-moving images for brains that are about under three or four years old, it wires your brain differently 'cause you have to constantly be refocusing.

Like, that wasn't a problem in my age 'cause the images didn't move that fast.

I mean, they, the commercials were 30 seconds long, not four seconds, right?

Um, but once you know that, don't you have an obligation to regulate it to, to protect the children?

Like, okay, well, then YouTube Kids can't have images changing two times in a second or two times in a minute or whatever.

Do an algorithm where you...

you know, if they detect the child doing this often-

It just, it slows.

Yeah.

...

like, it, it, it does some-...

the technology's-

Yeah.

...

there.

It's just, there, there's no...

we, you know, we talked about incentives before.

Right.

So, so, so I, I really think about incentive models, like the incentives must be misaligned.

They've gotta be misaligned.

Um, and we haven't mentioned once so far on all of our sessions the word liability, which I'm pretty proud about.

But at some point, like, what levers do you have left to regulate?

When you say regulate the outcome, is it through liabilities, through incentive structures?

Is it through, um, companies generally respond to incentive structures, they probably, they try to resist liability.

They try to externalize all the risks and privatize all the profit, um, and you know that.

So how do you regulate an outcome in that environment?

I think it's, it's...

now going back to kind of, because in, in the UK it's also a big debate right now about, um, social media and whether to ban or not in schools, um, I think there's obviously the question, just thinking about out- outcomes, right?

It's, it's making sure that, let's say, there is, that, that the new generations, that they have media literacy, that they are aware, that they can use and take and make the most out of what they have in terms of, let's say, access to information, right?

That's the outcome that we want, hopefully.

Um, the other thing is you...

if you ban, uh, social media or if you ban even kind of cell phones from the school, there will be a desire to use it anyway, because it is a core part of you as an individual t- being integrated in social life.

So then the question is, what are the other elements and other stakeholders, going back to what I was saying previously, then you need to think, okay, one answer is the education system.

The other answer is how do we sp- how do we then speak to the platforms and speak them a- their response also as an answer that we need.

So we need them to do more.

And so you start to pil- to kind of paint this almost like multi-stakeholder kind of set of responsibilities that need to be encompassed if you're thinking, and so the conversation is not about should we ban or not access to

smartphones in school.

Then the conversation is, as government, how can we incentivize a, a conversation about how to best use technology within the school system, right?

So that you don't shun the conversation from the school to then the home where sometimes parents might not be as, you know, interested in saying like, "You're going to have X amount of screen time."

Right?

At home is a different ball game.

So I think that's what I meant when in, when in...

thinking about outcomes, the outcome is we're going to get the best set of technology, the younger generations are going to do that, but the responsibility is spread across.

So we need measures for each of those, let's say, stakeholders.

And I think when it comes to this, when it comes to cyber and when it comes to like securing or safety of AI, it's also about thinking what are the designated responsibilities for these different stakeholders.

Three, two, one.

All right.

Thank you everyone.