In 2017, Ken Griffin’s Citadel started quietly assembling a small team of experts in machine-learning and natural-language thousands of miles from its headquarters in Chicago.
The team experimented with different forms of artificial intelligence designed to predict market moves in order to give traders and portfolio managers an edge, according to a person with direct knowledge. Using AI to predict what will happen in the market is a unique challenge — and one that Wall Street’s brightest minds are still trying to crack — because of the finite amount of data on major economic cycles. But it attracted top talent from tech giants like Microsoft and Uber.
The AI research team operated in Seattle and was led by Li Deng, who was poached after spending 17 years at Microsoft, where he led AI research. The team won over stock-picking portfolio managers, who rely more on human intuition to make bets in the market, as AI tools saved them time researching covered companies.
Some senior members of the team, including Deng, left in 2020. By the end of that year, the team was disbanded. A member of the team stayed and accepted a role in a different office, a person familiar with the situation told Business Insider.
“You have this kind of offshoot in Seattle that isn’t really that integrated with the business. Ken wants it to be, you’re paying these people a lot of money, you want them to then become increasingly more bought in and embedded within Citadel,” a former Citadel employee familiar with the matter told Business Insider. “I don’t even think they were particularly subtle about that. I think it was a pretty open dialogue that just wasn’t going to work.”
Interviews with 11 AI executives, recruiters, vendors, and consultants working on Wall Street suggest the culture clash at Citadel is symbolic of what hedge funds might face even as they use their deep pockets to lure in talent.
They painted a picture of what many technologists dream of: establishing an outfit and strategy to finally crack the enigma of using AI to beat the markets — and getting paid handsomely to do so. But inside, AI leaders can struggle to gain the trust of business leaders and break into investment teams. AI researchers have struggled with hedge funds’ culture around proprietary secrets, which goes against the open-source philosophy foundational to AI and academia.
Hedge funds are eager to harness AI and gain an edge. Some recruiters, including Michael Stover, are seeing double- and triple-digit growth in AI hiring among hedge funds and prop-trading firms.
“If it’s not already, it’s going to be one of the next great wars for talent,” said Stover, the head of Americas at The Omerta Group, a financial services executive-search firm.
Some of the biggest funds have been building out their AI teams with key hires in recent months. Millennium Management hired a global head of AI from Bloomberg in October, while ExodusPoint poached its head of artificial intelligence from Balyasny Asset Management in September. Man Group and Balyasny promoted executives last fall to lead new teams dedicated to applying AI firmwide.
In some cases, hedge funds are paying AI researchers two years out of university between $400,000 and $500,000 in total compensation, said Agni Ghosh, a director at the recruitment firm Stott and May. For heads of AI, he said, hedge funds are dishing out a minimum of $1.5 million in total comp.
A senior machine-learning exec told BI that at one large hedge fund, “curiosity is through the roof, acceptance is improving, but I still feel like there’s not a lot of appreciation” for how AI shows up in portfolios and moves the portfolio-building process.
To be clear, the AI technology isn’t completely new to hedge funds. Firms have long weaponized quants who use deep math expertise, machine learning, data, and so-called predictive AI to make bets in the market. But the introduction of generative AI, which became mainstream thanks to ChatGPT, has opened up new opportunities.
These days, AI is all about increasing efficiency, such as automating grunt work for analysts or alerting portfolio managers about market-moving changes in covered companies. AI can also help out in the middle and back offices, where operations are still largely manual and paper-based, said Sumeet Chabria, a former Bank of America tech exec who now runs an AI advisory firm called ThoughtLinks. Some firms are handing the investment process to AI, including understanding financial and economic patterns and creating and testing investment theses.
But perhaps the top reason hedge funds are interested in the technology is that AI capabilities are bait in the war for investing talent.
“These hedge funds are now in a space race to offer generative AI and unstructured-data capabilities as part of their honey trap for why portfolio managers should come to them,” said Conor Twomey, the head of customer success at KX, an AI vendor that works with many Wall Street firms.
Stover, who recruits PM and tech talent for multi-strategy and quantitative hedge funds, said the PMs he works with are saying more often now than at any other point in his 26-year career that the reason they’re leaving their funds is a lack of tech investment.
On Wall Street, hedge funds’ AI strategies are usually dictated and carried out by a centralized group of experts who decide what is built for the entire firm.
While this helps mitigate the risk of different teams unknowingly doing the same work, it can create a disconnect with portfolio managers that impedes an AI team’s ability to embed themselves in the rest of the organization. PMs typically have a degree of autonomy over their small teams, and it may not be obvious to them how to implement the tech in their trading processes.
“It can’t be just driven by a small group of technologists.”
Tim Mace, a managing director and head of data and machine learning, a new division leading AI for the London-based Man Group, said the challenge is not just integrating AI with the firm’s systems, but also fitting in with the way people work across an organization.
“They need to understand how this stuff works if they’re going to get value from it,” Mace said of AI. “It can’t be just driven by a small group of technologists.”
Political jockeying has become part of these leaders’ remits. The AI lead needs “engaged sponsors,” or PMs willing to advocate for the team, said a San Francisco-based recruiter who places senior tech execs for Wall Street firms. Having a good internal reputation is everything, they added.
An executive who leads AI at a large hedge fund described the goal as “partnering with other teams internally who can kind of be advocates on our behalf.”
That’s in part because AI leaders’ pay can be tied to the output of their technology, Stott and May’s Ghosh said. But getting these stakeholders interested isn’t as easy as pinning a sign-up sheet to the wall.
“You actually need to do some sales as well,” the San Francisco-based recruiter said, “because some may not be believers or some may not even know that it’s a useful service that can be provided.”
The recruiter said that if firms take too long to drum up internal interest, they “won’t be able to attract the right kind of talent.” Top talent, they said, is often less inclined to join a firm that seems as if it’s just experimenting with the tech.
It’s no secret that hedge funds keep things close to the vest. For AI researchers moving from Silicon Valley to Wall Street, that could lead to problems, according to the former Citadel employee who witnessed turnover across different funds.
“Intrinsically that may not stack up with the person’s satisfaction, no matter how much money you pay.”
Tech giants like Google and Meta and trailblazing AI firms like OpenAI and Anthropic encourage research-engineering talent to publish research and promote open-source development, which, unlike proprietary technology, is available for developers to use, learn from, and contribute to. The researchers like it that way, too, because it gives them a chance to build a brand among the academic community and to collaborate with revered professors, labs, and scientists.
“If you go to a hedge fund, the trade-off is you’ll probably be paid not to do that essentially, and therefore intrinsically that may not stack up with the person’s satisfaction, no matter how much money you pay,” the former employee, who remains in the industry, said.
Sometimes the work itself can lead to burnout. The nature of the research can mean people are working on the same problem for a long time, and that can lead to retention challenges.
Highly qualified and educated researchers who work within smaller investing pods focused on one asset class “get bored pretty quickly,” they added. It’s not uncommon for someone in that role to get pigeonholed into working on the same problem with the same data.
Successful AI leaders will need to walk a fine line. They’re brought in to be a change agent, “but it comes at a cost of potentially overstepping, potentially going further than the company is ready to go,” Peter Finter, the chief marketing officer at KX, said.
“They have to be both the conscience of the organization, ensuring that they’re not overstepping their bounds, but also the champion of change driving into this new space, which is relatively unexplored,” Finter said. “It takes a very special person to be able to sit on both sides of that.”
These days, you probably won’t see Citadel advertising AI researchers on its careers page. It’s one of the few large US hedge funds without an executive or centralized team dedicated to AI or machine learning. That’s not to say the firm isn’t doing anything with the technology — a source familiar with the matter said AI talent is infused throughout the business and that Citadel is still investing in AI.
Hedge funds often use more-general titles like quant researcher or deep-learning researcher to try to protect internal organization and prevent their talent from being poached.
But Citadel’s attempt to embed a Seattle-based AI team into the firm over about 7 years ago highlights that even the world’s most successful hedge fund “experimented with it and it didn’t work out,” the former Citadel employee said.
For hedge funds wanting to get the most out of their AI talent, Ghosh said, “you’ve got to be able to bring to life the working culture and an environment that these types of profiles can fit into, as well as keeping your identity.”