Read in browser

Happy Anthropic S-1 day Here’s the draft prospectus for Anthropic PBC’s initial public offering. In writing a prospectus for a frontier artificial intelligence IPO, you have to make choices about what to promise in your good vision of AI, and what to threaten in your bad vision. SpaceX went public this year promising “the largest actionable total addressable market (‘TAM’) in human history,” $28.5 trillion, which is a lot, sure, but still a finite number of dollars. OpenAI has raised money to build artificial general intelligence with the considerably more abstract promise that “it may be difficult to know what role money will play in a post-AGI world.” So the two benchmarks are something like “we will make more money than any company has ever made before” and “we will make money obsolete.” Anthropic is, by these standards, relatively sedate. From its founders’ letter: We encourage you, as a prospective owner of our stock, not to think of your purchase in purely transactional terms. We hope you will see it instead as an opportunity to be part of something greater. The advent of machine intelligence is one of the seminal moments in human history. As one of the companies at the very center of that moment, we have a rare opportunity to shape it for the better. Our aim is to help humankind chart a safe course through this technological evolution, open new frontiers of discovery, and bring forth an age of miracles. Everyone who owns our stock, whether an institution or an individual, becomes part of that effort. We hope you will share our ambition to see it through. “Buy our stock for miracles” is pretty modest as AI hype goes. But it wouldn’t be an AI prospectus without threats. I searched in vain for words like “extinction” or “kill everyone,” but there are a couple of mentions of “existential risks to humanity.” The prospectus does not quite say “our products pose an existential risk to humanity.” It attributes that belief to someone else; it says that some people think Anthropic might pose an existential risk to humanity: Concerns have been raised both within our organization and by other researchers, regulators, policymakers, and industry participants that sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy. We have observed that our models are contributing to the pace of our own research and these risks will increase as AI systems substantially automate AI research and development if, for example, they accelerate AI progress beyond our ability to evaluate and safeguard new models, which could adversely affect our business, financial condition, and results of operations.  I have to say that, if the robots do kill everyone, I am going to spend my last few seconds on earth being annoyed at that use of the passive voice. “Concerns have been raised … within our organization … that sufficiently advanced AI systems could … pose catastrophic or existential risks to humanity,” ah. Here’s an essay that Anthropic Chief Executive Officer Dario Amodei wrote this year saying things like “it is clear that, if for some reason it chose to do so, [powerful AI] would have a fairly good shot at taking over the world (either militarily or in terms of influence and control) and imposing its will on everyone else” and “more capable LLMs … might be capable of … researching, and potentially creating, a dangerous new type of organism … [that] could proliferate in an uncontrollable way and crowd out all life on the planet, in the worst case even destroying all life on earth.” Concerns have been raised! Over the weekend, Bloomberg published an essay by Tom Orlik, Michael Deng and Dongmiao Zhang titled “Anthropic’s Pitch to Investors Starts With Taking Their Jobs”: Buy our IPO: We will use the funds to take your job and maybe kill you doesn’t immediately sound like an attractive offer. In a strange way, though, it has a perverse appeal. Smart investors understand the importance of hedging against risk. If you can’t beat our new AI overlords, why not join them? Maybe use the profits to build a bunker. That’s right, right? The biggest risks in Anthropic’s prospectus are not really risks to its business; they are risks that its business poses to everyone else. The one I quoted above shrugs into its conclusion that killing everyone on earth “could adversely affect our business, financial condition, and results of operations,” but surely that’s not the main problem with killing everyone on earth. Or another risk factor says: “As AI systems become more capable or prevalent, public perception that our models, platform, and applications contribute to job displacement or economic disruption could generate significant negative media coverage, customer or employee backlash, or organized opposition, regardless of the actual impact of our models, platform, and applications on employment.” The explicit content of that one is “we might get negative media coverage and customer backlash,” but its implicit warning is “AI might take everyone’s jobs.” In the maximalist model of AI adoption — the one in Anthropic’s risk factors — all other human enterprise will be eaten by Claude, so you might as well own some Claude. The more dangerous Anthropic is to everyone, the more important it is to be on Anthropic’s side. Anthropic governance On the one hand, Anthropic is a trillion-plus-dollar company that is looking to raise billions of dollars in an initial public offering so it can continue pouring money into building and training frontier artificial intelligence models. On the other hand, Anthropic is a weird effective-altruist worker cooperative that worries about AI safety and does not want to be sullied by money. You might lay out its hierarchy of control — its hierarchy of who should get a say in its big decisions — like this: Humanity. Anthropic is a public benefit company whose “purpose is the responsible development and maintenance of advanced AI for the long-term benefit of humanity.” If it ever contemplates doing something bad for humanity, it should stop.  Its founders. Anthropic has seven co-founders and a famous origin story in which they left OpenAI to start a more safety-oriented AI lab. Those founders can probably be trusted to make good decisions for humanity, most of the time, though they are second on this list because they want some safeguards on themselves. Its employees. Anthropic’s employees are also pretty safety-oriented and smart about AI, and you could imagine scenarios in which they should overrule the billionaire founders, because they are closer to the risks or because they are less influenced by money. Regular shareholders. “Our aim,” says Anthropic’s prospectus, “is to help humankind chart a safe course through this technological evolution, open new frontiers of discovery, and bring forth an age of miracles. Everyone who owns our stock whether an institution or an individual, becomes part of that effort.” The shareholders are helping with the miracles too! Big tech. Anthropic has raised billions of dollars from Amazon and Google and has commercial relationships with them. They are technically shareholders, but not the miracle-working kind. They are big profit-seeking tech companies whose commercial interests might conflict with Anthropic’s long-term-benefit-of-humanity purposes. And in fact Anthropic has a complicated governance structure, with five classes of common stock, reflecting that hierarchy: Humanity (Class T). An independent trust, the Long-Term Benefit Trust (LTBT), has the power to appoint and remove four of the seven members of Anthropic’s board of directors. The trust consists of “five disinterested voting trustees” who are paid $400,000 a year and who are prohibited from owning any Anthropic stock; their job is to make sure that Anthropic follows its mission of building AI for the benefit of humanity. (They are listed on pages 245 to 247 of the prospectus.) So a majority of Anthropic’s board of directors is appointed by the long-term benefit trust, not by shareholders. Well, technically the trust is a shareholder. It owns one share of “a special class of common stock, Class T,” which has no economic rights but gets to appoint four of the seven directors. The founders (Class F). The founders get to appoint the other three members of the board of directors. They jointly own a separate company, called the Founder LLC, which also owns one share of a special class of non-economic Anthropic stock, Class F. Technically the Founder LLC votes with other shareholders to elect the other three directors, but “our Class F common stock is entitled to a fixed 50.1% of the total voting power of our outstanding shares of capital stock,” so the founders’ chosen candidates will always win. So four board votes for humanity, three for the founders. The founders also care about humanity, though. “While the LTBT, Founder LLC, and our co-founders share a long-term commitment to our mission that may benefit stockholders who share our public benefit goals, the separation between voting power and economic interests may result in such persons exercising their influence in a manner that prioritizes actions or outcomes they believe to be consistent with the Company Purpose and the best interests of those materially affected by our conduct over the pecuniary interests of our other stockholders.” Employees (Class E). Actually the trust can appoint three directors however it wants, but its fourth director has to be approved by the board’s nominating committee. If the nominating committee “does not concur with [the trust’s] nomination [or] removal” of that fourth director, then Anthropic’s employees get to vote on the director. I suppose the theory is that the trust is a disinterested group of experts pursuing the long-term benefit of humanity, but it is not close to the coal face of AI; the founders have devoted their lives to Anthropic and presumably also care about the long-term benefit of humanity, but they also have billions of dollars of personal wealth at stake in Anthropic’s decisions. If the founders and the trust disagree, the employees get to decide who’s right. The way this works is that the employees have their own special class of stock, Class E, which has no economic rights but gets to vote to break these board ties. Employees get one share of Class E stock for every six months they’ve worked at Anthropic, plus one extra share for every share of regular (Class A) stock that they’ve pledged to donate to charity. So the more public-spirited the employees are, the more say they get. Regular shareholders (Class A). People who buy shares in Anthropic’s public offering get Class A stock. (Employees also get Class A stock: The Class E is for voting, the Class A is for economics.) The Class A stock has one vote per share, though it will always be outvoted by the founders. But, unlike the previous three classes of stock (T, F, E), it has economic rights. If you own a Class A share, you get your pro rata share of Anthropic’s trillion-plus-dollar valuation. Big tech (Class N). Amazon and Google get Class N stock, which is equivalent to Class A stock economically, but which has no voting rights.[1] (If they sell the Class N stock, it generally converts into Class A voting stock.[2]) Amazon and Google rank below the regular shareholders, presumably to reflect their connection to filthy commerce. (Also Anthropic has pledged to give away at least 5% of its stock to a “public benefit fund”; that will also be Class N.) I guess you can just do this? We talked last week about the traditional method of maintaining founder control at a tech company. It’s just: The founder(s) get special stock with 10 votes per share. The math on that is that, as long as the founders own at least 9.1% of the company,[3] they will have a majority of the voting power and will be insulated from outside shareholders. There’s no magic to these numbers; they are just a rough way to balance giving the founders an extra say over the company they created with making sure that they have a reasonable amount of skin in the game. But if you’re really concerned about AI risks, that is probably insufficient, in two ways. On the one hand, if Anthropic needs to raise a gazillion dollars to safely build AI, there’s no guarantee that its founders will continue to hold at least 9.1% of its economic value. Divorcing governance from economic value entirely — giving the founders majority control no matter how much stock Anthropic sells — is a way to ensure that the trustworthy public-spirited founders will always keep control of their robot army. On the other hand, how much do you trust those founders? The “robot army” line in the previous paragraph was a reference to Elon Musk, who has said “I just don’t feel comfortable building a robot army” at Tesla Inc. without significant voting control, and who will probably consolidate that voting control by merging Tesla with SpaceX. Elon Musk surely trusts himself to command a robot army, and many Tesla and SpaceX shareholders probably agree, but not every outside observer might. Giving a nonprofit board — made up of non-billionaires with no economic ties to the company — the power to supervise the founders’ command of their robot army might help a little, though of course there’s no guarantee that the nonprofit trustees will do the right thing either. Maybe they and the long-tenured employees will team up to seize control of the robot army. The trustees have no economic interest in Anthropic’s stock, but money isn’t everything, and it may be difficult to know what role money will play in a post-AGI world. It is possible that I am overthinking this. This is the governance structure that you would put in place if you worried about existential AI risks, but it is also the governance structure that you would put in place if you ran a fun quirky little mission-driven worker cooperative that happened to need to raise hundreds of billions of dollars of outside capital. “Sure we’ll sell stock,” you’d say, “but we don’t want to sell out. We want to keep our friends in charge and not do anything that would make our jobs less fun or more businesslike.” This is the way to run a workers’ co-op at a public company. Fake polls Is this … commodities fraud? The pollster had been releasing surveys since June, shifting from one midterm battleground to another: Michigan. Maine. Kansas. On Tuesday night, after the polling website FiftyPlusOne contacted the pollster, Wedgewood Polls, with questions about the authenticity of its surveys, the team behind the supposed data firm acknowledged that almost all of its work was fake. The whole thing, Wedgewood claimed, was an “experimental project.” The fake polls were highlighted by well-known polling aggregators, including on the pages The New York Times uses to track surveys published by outside pollsters in key midterm races. Nate Silver writes that “compared with past fake pollsters — which often tried to fit in rather than draw attention to themselves — the Wedgewood ‘polls’ were a bit more narrative-shaping,” showing results that did not match the consensus of other polls. Byrne Hobart writes: They say it was a social experiment, but if you can modify the results of an election prediction, you can modify the prices of the relevant prediction market, so this kind of thing, in addition to being easier to pull off thanks to AI, is easier to monetize through prediction markets. Prediction markets are an information aggregation tool, but they’re also a bug bounty for finding situations with a high return on agency, and a lot of that will be the kind of agency that imposes negative externalities on everyone else. Like, you could definitely have made tens of dollars on Kalshi, perhaps more, by shifting the aggregate polls for the Maine Senate race by a few points. Did they? Was that the monetization here? I have no idea, and my guess is no. But: Surely, if they did use these polls to trade on Kalshi, that would be illegal commodities market manipulation (not legal advice), and Even if they didn’t, if you bought election contracts on Kalshi at prices influenced by these fabricated polls, wouldn’t you be annoyed? Wouldn’t you be tempted to sue for damages? Might the US Commodity Futures Trading Commission be interested in this? Aren’t they intentionally publishing fake data about the prices of commodities that support large derivatives markets? Where the “commodities” are congressional races? It feels like a lifetime ago, but as recently as 2024, congressional elections were not commodities that traded on regulated US futures exchanges, and the CFTC was trying to keep it that way. We talked in May 2024 about a proposed CFTC rule to ban election event contracts; the then-CFTC chairman said: Allowing these contracts would push the CFTC, a financial market regulator, into a position far beyond its Congressional mandate and expertise. To be blunt, such contracts would put the CFTC in the role of an election cop. The CFTC’s jurisdiction as mandated by Congress and solidified in our statute, the Commodity Exchange Act, recognizes our expertise in markets for goods, services, rights, and interests—which can include events associated with financial, commercial, or economic consequences. We are tasked with upholding the public interest by ensuring that America’s derivatives markets provide a means for managing and assuming price risks and providing for price discovery through liquid, fair, open, transparent, and financially secure trading facilities. Market integrity is featured so prominently within that mandate that the CFTC has civil enforcement authority when it comes to the potential for fraud, manipulation, and other abuses such as the dissemination of false information in the underlying or commodity cash markets. Political control contracts on CFTC-regulated exchanges would push the CFTC far beyond this historical expertise and jurisdiction, and potentially place the CFTC in the position of monitoring such markets for fraud and manipulation in elections themselves. That is: The CFTC is primarily in the business of regulating, say, wheat futures, not wheat. If a farmer sells wheat to a baker, the CFTC is not involved. But if a farmer is doing “fraud, manipulation, [or] ... dissemination of false information in the underlying … markets,” the CFTC can go after her: The CFTC’s job of policing commodity futures markets is so important, and those markets are so obviously tied to their underlying commodities, that the CFTC does police fraud and deception in the underlying markets. Here, of course, the futures markets are election betting markets, and the underlying markets are elections.[4] And polls, and “dissemination of false information” about elections. If you put up fake polls, that’s now a problem for the CFTC. Now the CFTC is an election cop. One other point. The general idea is that prediction markets are truth machines. There is some data in the world about, say, elections; there are polls and focus groups and fundraising totals and economic fundamentals and whatever else. There are people, like Silver, who are in the business of aggregating all of that data into a single prediction of the thing you really want to know, which is “who will win the election?” And, the theory goes, a prediction market will be a particularly good aggregator: It harnesses economic incentives and the wisdom of the crowd to get an efficient-markets prediction of who will win. The prediction market is a zero-sum venue for predictors to gamble, but it creates a positive societal externality: Even if you never trade on a prediction market, you can go to its website and see who is likely to win an election, which might be useful information for you. But that’s not necessarily true in general equilbrium. If the prediction markets get big enough, they can become untruth machines; they can make it harder to know the truth (here, who’s going to win). Prediction markets create incentives to generate fake data and put it out into the world; people warm up airport temperature sensors or create fake Spotify streams to win large bets on otherwise unimportant facts. Or on important facts like elections. Prediction markets are a good way to aggregate the existing data in the world and turn it into predictions, but they also create incentives to put fake data out into the world to influence the predictions. Things happen Meta Seeks Up to $95 Billion in Equity to Fund AI Spending. SpaceX Agrees to $40 Billion Chip-Financing Deal With Apollo. Plan for AI-run companies challenged by US lawmakers. Two AI Chipmakers Make Up 34% of America’s Earnings Boom. The reclusive quant billionaire coming into the light. SpaceX IPO lockup drives surge in controversial derivatives trades. Why So Many Shoppers Think Retailers Are Using AI to Jack Up Prices. Anthropic Sued for Training Claude on Elvis Presley Songs. Kalshi Courts Wall Street in Bid to Be the ‘Inflation Exchange.’ In San Francisco, a Family Earning $325,000 Qualifies for Government Aid. [1] A few years ago, we discussed OpenAI’s governance structure; it also featured (1) a nonprofit board that had final say over decisions and (2) a giant pile of money from a big tech company, in its case Microsoft Corp. The nonprofit board fired OpenAI’s chief executive officer, and then he came back in part because Microsoft, which had no voting rights, was like “what the heck?” I suppose Anthropic wants to further formalize that its partners have no voting rights. [2] With don’t-get-cute exceptions: It doesn’t convert if the buyer is a Class N holder, or if the buyer would be above 9.99% voting power. [3] You know, 91 founder shares with 10 votes each (910 votes); 909 public shares with one vote each (909 votes). [4] Likewise, ceteris paribus, for sports. If a baseball player throws a game, commodities manipulation! Follow Us You received this message because you are subscribed to Bloomberg’s Money Stuff newsletter. If a friend forwarded you this message, sign up here to get it in your inbox. Unsubscribe Bloomberg.com Contact Us Bloomberg L.P. 731 Lexington Avenue New York, NY 10022 |