Anthropic 称已阻止 Claude 被用于生物或化学武器相关用途

I haven’t been sure what to make of the report that this Science piece is referring to. Anthropic says that it has blocked efforts to use its Claude AI systems for a number of undesirable functions, including possible design of bioweapons. One the one hand, it is unfortunately plausible that sufficiently powerful systems could be used in this manner, and that’s really something that we would want to avoid. On two of the other hands, though, it is in Anthropic’s interest both to illustrate how powerful its own software is said to be, and to demonstrate that they’re aware of possible misuse cases and have been trying to mitigate them.

So were these indeed such cases? As you’ll see from the Science article, not everyone agrees on that. Three of the five cases involved modifying viruses (chikungunya, avian influenza, and some unspecified member of the orthopoxvirus family), but it’s hard to tell if the AI requests were directed at nefarious modifications or simply at the sorts of things you’d do normally in a research project. This uncertainty is the sort of thing that led (and continues to lead) to all kinds of confusion in the lay public about “gain of function” experiments in general. 

Although it might sound counterintuitive to people outside the field, looking at mutations that are known to increase infectiousness, virulence, stability and other undesirable qualities in a potential pathogen are often the best ways to figure out how it’s accomplishing these improvements, what to be looking out for in the wild, and what the best ways might be to counter these changes. You would certainly want to be doing these things in an appropriately secure laboratory - that should go without saying - but if you try to be Extra Safe by banning any such work at all you will almost certainly end up more vulnerable and less prepared than before.

Some of the ways to distinguish the good uses from bad would be seeing if the proposed mutations are things that have already been found in sequences that are already in nature, but perhaps combined in ways that have not surfaced yet. (That’s as opposed for fishing for completely new mutations thought up in the lab). Or on further inspection, whether these are being introduced into fully competent viral strains or into otherwise-weakened ones (or into “pseudoviruses” that have had their ability to reproduce stripped from them entirely). Now to be sure, Anthropic says that some of these requests were coming in through VPNs and some used what appeared to be deliberately vague language, suggesting that there may have been some attempts to disguise intent. But even those don’t establish wrongdoing. 

But it’s true that (even if these attempts weren’t anything to worry about) that future versions of these tools could be used to do harm. And that’s true of any new technology. I’ve been low-level worried about this since well before the days of AI assistants, and their advent definitely doesn’t make me any less worried. That said, designing a horrible pathogen is - thank God - a whole lot harder than it looks. As anyone who has had to go through multiple iterations of just some benign make-this-protein construct in some model organism can tell you, these biological systems are often finicky and refuse to do what you want them to do, for reasons that are extremely unclear and difficult to troubleshoot. And that applies to evil ideas as well as good ones: biology is hard.

Honestly, I’m more concerned about bootstrapping nefarious chemistry ideas for now, because (although it pains me to admit it) organic chemistry, steaming pile of craziness that it is, is still a lot more straightforward than molecular and cell biology. But at the same time, we should remember that the basic synthetic methods for very effective toxic agents (nerve gas an more) are comparatively well-known and have been so for decades. That’s not to say that there haven’t been some lunatics giving these the good ol’ sociopathic try (Aum Shinrikyo!) That article, if you continue reading it far enough, will make your eyebrows crawl all the way across the top of your head - the 1995 Tokyo subway nerve gas attack was only one of their attempts and only one of their chemical ideas. Another difference between attempts at chemical weapon development versus biological is that there are far fewer points of overlap or confusion between "honest" use cases and bad ones: there are all sorts of reasons to look at viral mutations, but there are far fewer explanations for being interested in the practical synthetic details of (say) phosphonofluoridates.

So the deranged killers are indeed out there, but they have not been as much of a problem as they perhaps could have been so far. For that I am grateful, but at the same time there is no reason to believe that this is a natural state of affairs that will continue without anyone bothering to check up. It would be convenient if the sorts of people who think that these plans are worthwhile would also be technically incompetent enough to make themselves the first victims along the way, but we can’t count on that. For now, I think we have to continue to be watchful, and that “we” must definitely include the companies developing these oh-so-helpful software companions.

注意转载授权 · 阅读原文