Experts interviewed for this report believe machine learning will be mildly disruptive to financial-market operations, at least in the next three to five years. When it comes to trading, Babak Hodjat, Chief Scientist at Sentient Technologies, a firm which develops AI software, predicts there will be a sizeable impact but not fundamental disruption. “The reason is that you’re moving from people and quants coming up with the strategies and deploying them to the machines coming up with the strategies and deploying them. You can churn through that faster but the end result will be similar.”

Most of the experts agree, moreover, that the technology will have many positive applications. Paul Ebner, a senior portfolio manager within BlackRock’s Scientific Active Equity Unit, believes trading will benefit from the depth of analysis machine learning tools enable over a wide breadth of companies. “It’s being able to go a couple of steps deeper than you could just by using, say, data in a spreadsheet. Speed matters but it’s a different kind of speed than high-frequency trading. For us it’s being able to process a lot of data very quickly and coming up with the right answer that the markets will eventually discover.”

BlackRock’s Scientific Active Equity unit – a roughly 100-member team which includes data scientists and machine learning specialists as well as more traditional financial-industry ‘quants’ – is putting machine learning techniques to work in different ways to forecast share price movements. According to Mr Ebner: “We’re applying tools to analyse data about companies and using that data to forecast the fundamentals, and then ultimately to forecast their stock returns and construct portfolios around that.” An example is the team’s analysis of companies’ currency exposure to gain more in-depth insight into their expected earnings well before their quarterly announcements (see forex exposures case study below).

Survey respondents are clear about another benefit machine learning will bring: 64% believe its use will have a positive effect on competiveness in financial markets (see chart 3). At first glance, this finding seems counter-intuitive. The high costs of the best talent and the most advanced technologies should make AI accessible to only those with the deepest pockets. This is true to an extent – only a firm like Bridgewater Associates could afford to hire the chief engineer behind IBM’s legendary supercomputer, Watson.

However smaller companies, and even individuals, have proven that they can be at the forefront of new innovations. This is partly because open-source software empowers smaller organisations to experiment with advanced algorithms and code. A recent example was seen in March 2016 when two retired hedge fund “quants”, with no prior experience of working with AI, managed to design an algorithm that diagnoses heart disease from MRI images. These individuals were relatively inexperienced in AI and yet they were able to design a transformative application by downloading software from the open-source site, GitHub.

What this means for competition across financial organisations, only time will tell. However, whilst industry giants, such as BlackRock, will be doing a lot of the heavy lifting of research and pilot work, AI applications and services could become widely available for a range of small and medium sized organisations.

What could go wrong? Plenty, according to Andrew Lo. He believes the markets may be in for more flash crashes, for example, or for other negative developments about which neither the industry nor regulators currently have a clear understanding. “The nature of these strategies makes them very difficult to understand. That means that the interactions are going to be hard to predict, in the same way that nobody predicted the flash crash of May 6, 2010 – even today we still don’t really understand what happened.” He also points to the demise of Knight Capital, a major US trader whose software glitch in 2012 caused it US$440m in trading losses and sent it to the brink of bankruptcy: “I don’t think we’ve nearly fixed those kinds of issues, because ultimately you’re dealing with a mismatch between human ability and technology. It’s Moore’s law meets Murphy’s Law.”

Technology will not be able to remove the risks inherent in some financial activities, such as making bets on future events. These are likely to persist, regardless of whether humans or algorithms do the work.

As Arun Srivastava, Partner at law firm Baker & McKenzie argues, "Financial institutions have been fined billions of dollars because of illegality and compliance breaches by traders. A logical response by banks is to automate as much decision-making as possible, hence the number of banks enthusiastically embracing AI and automation. But while conduct risk may be reduced, the unknown risks inherent in aspects of AI have not been eliminated."

Regulators are uncertain as yet what the risks of machine learning are but are focusing on algorithms as an area where problems could occur. Victoria Pinnington, Senior Vice President of Market Regulation of the Investment Industry Regulatory Organization of Canada (IIROC), says her greatest concern currently is around the crafting of algorithms, in both machine learning and broader systematic trading contexts. “If there is a problem with the algorithm,” she says, “the impact on the markets could be considerable.”

Algorithms can malfunction in a variety of ways. One of the most common errors is known as “overfitting”, which usually occurs when an algorithm is overly complex and lacks a suitable hypothesis. In such cases they fail to distinguish useful correlations (signals) from the mass of irrelevant data (the noise) and instead identify “phantom” factors or specious correlations. Imagine trying to record a classical music concert with a sensitive microphone. Overfitting would equate to picking up the surrounding background noises, rather than the sound of the orchestra. As Babak Hodjat warns, “people could mismanage machine learning and not do the validation. If you take a machine learning algorithm and not sufficiently validate it, you might have something that’s overfitting, that might look pretty good right now but might fail miserably tomorrow. More scrutiny is required there.”

The risk of programming error increases with the pressure to launch new programmes. As mentioned earlier, differentiation is crucial to the success of an AI trader. Being the first successful application gifts an organisation a unique and profitable opportunity – an uncontested marketplace. Consequently there is a risk that organisations will rush their strategy to market. As Nate Soares, a former Google engineer and current research fellow at the Machine Intelligence Research Institute recently estimated in an interview with The Financial Times, “there is only a 5% chance of programming sufficient safeguards into advanced AI.”

There is a great deal of uncertainty among survey respondents as to whether organisations understand the legal risks associated with new financial technologies: 47% are not confident that they do (see chart 4). Mr Price is not surprised. “The reason is that this technology is at a nascent stage, and it is evolving. The fact that people are cautious and a little unsure about what some of the risks might be, reflects that nascent stage of where the technology's at.”

One risk is corporate liability. Flawed investment decisions could be made as a result of poor data, erroneous analysis about company performance, or malfunctioning algorithms, which could cause investors significant losses. Liability could also arise should machine learning models make flawed decisions about credit risk: financial losses could occur to lenders, or alternatively borrowers’ reputations could be damaged. There is also a lack of clarity about which parties would bear liability should such situations occur –the financial institutions themselves, the writers of the algorithms, the exchange platforms, data providers or other parties.

The intelligent, data-crunching properties of machine learning may also take data protection and privacy risk to another level. Personal investor data or sensitive company data falling into undesired hands, whether by accident (to cyber hackers) or design (to marketers and governments), is by now an all too familiar risk of the Internet age. This risk will grow simply by virtue of the much larger volumes of data machine learning models will gather in. Organisations will increasingly need to understand how data privacy is entwined with laws on consumer protection, as well as wider pieces of legislation such as the EU Cookie directive. Adrian Lawrence, Partner at Baker & McKenzie expects access to data to play a central role in the scope and impact of AI systems, noting that "data, and the various rules and processes which both enable and regulate access to and use of that data, stand at the heart of disruptive fintech businesses. Even the most advanced and intelligent algorithms and models are useless without efficient, secure and legal access to detailed, accurate and up-to-date data sets."

Beyond legal risks, the survey respondents clearly lack confidence that the impact of AI is fully understood by their organisations. 49% of respondents are not confident that their organisation understand the other material risks associated with AI. Only, 32% are confident. Given the early-stage development of applications, this finding indicates that AI will present organisations with a set of risks, most of which are still to be defined.

The biggest risks, according to some experts, lie less in machine learning techniques themselves, but rather in humans’ misuse of technology, or misplaced confidence in technology to achieve goals by itself without human guidance. “If we have blind faith in technology,” warns Babak Hodjat, “things will go wrong. If the success of AI means more use of technology in an uncontrolled and non-principled way, then we’re risking more.”

Saeed Amen worries that the industry will use machine learning as a sort of black box. Should this be the case, he says, “they’ll end up creating a trading model that they don’t really understand the ins and outs of. That is a dangerous scenario but it’s the same with any systematic model. You really need to understand what’s going on in the trading strategy.”

Just like humans - programmes, computers and machines have the capacity to be stupid. The danger is that they can act at a far greater scale and speed. Examples such as the Knight Capital disaster, serve to illustrate the importance of maintaining human oversight, comprehension and control of emerging AI systems.

Herein lies the contradiction at the core of the technology. When confronted with inherently risky tasks – such as making investment decisions and bets on unknown future events – over-reliance on AI can magnify systemic risks. Yet the same technology can improve the depth and quality of financial institutions’ due diligence of companies. Through their powerful data-crunching capabilities, such applications can also help identify fraud, money-laundering, bribery and other corrupt practices that more conventional methods would struggle to uncover.

With these examples in mind, the survey respondents appear hopeful that machine learning will help them to minimise risks, in some cases. Nearly six in ten (58%) believe it will “greatly enhance” their risk-assessment processes. Machine learning techniques can, for example, be used to alert fund managers about emerging weaknesses in invested companies (see section above “Understanding forex exposures to predict earnings”).

Astrid Raetze, from Baker & McKenzie also expects consumer credit risk assessment to be enhanced through more thorough risk profiling of customers. Ms. Raetze suggests that, “AI should also reduce risk in some areas if deployed properly. Market misconduct, anti-money laundering (AML) and Know Your Customer (KYC) processes are areas where regulators could harness AI to improve regulatory oversight and scrutiny.” Machine learning-based analytics can also identify patterns in client activity that may point to some form of malfeasance. This helps explain why respondents point to risk assessment, ahead of other areas of operation, where they expect machine learning to be implemented over the next three years.

Survey respondents appear to agree that algorithms need more regulation. Over half of them (56%) say that affording regulators’ access to examine trading algorithms would help to keep the financial system safe. This is a telling result that comes at a time when a number of financial regulators are planning to make the source-code of algorithms open to examination by authorities.

For example the US Commodity Futures Trading Commission (CFTC) is currently trying to push forward the regulation of Automated Trading (otherwise known as “REGAT”). One of the most controversial features of this proposal is enabling the CFTC and the US Department of Justice access to a financial firms' trading algorithm.

REGAT’s most vocal opponent, Chris Giancarlo, (a Republican member of the US CFTC) has argued that giving regulators this degree of control will impose extra compliance costs on smaller market participants and hamper innovation in the futures market.

Moreover, Mr. Giancarlo, is primarily concerned that this regulation would mark an unprecedented invasion of private intellectual property rights by public authorities. As he recently expressed in a statement, “I am unaware of any other industry where the federal government has such easy access to a firm’s intellectual property and future business strategies. Other than possibly in the area of national defense and security…” Similar mechanisms of oversight are expressed in MiFID market regulations in Europe and could lead to similar conflicts over the intellectual property of algorithms. Indeed, John Flaim at Baker & McKenzie predicts a significant change in the nature of intellectual property disputes, arguing, “Intellectual property ownership will be key. There has been a demonstrable increase in the filing of e-commerce patent applications since the turn of the century. However the U.S. Supreme Court issued a decision which has led to the sweeping invalidation of many e-commerce patents in the courts in 2014. Many other jurisdictions, like Europe, do not permit patenting of software at all. The financial industry is thus grappling with balancing their increased research and development efforts with a higher bar for patenting in the e-commerce space.”

Before source code repositories are handed over, regulatory agencies also need to demonstrate competency on data protection and cyber security. In March 2014 a group of Chinese hackers, managed to hack the US Office of Personnel Management. They stole the records of 21 million US federal employees, including senior members of the US CFTC. Vulnerabilities like these need to be repaired before authorities gain access to algorithms, yet their occurrence lead some to suspect that regulatory authorities are ill prepared for rapid changes in technology.

Over time, machine learning will almost certainly push some human beings – traders, analysts and other industry employees – out of their existing roles. Within 15 years, 68% of survey respondents expect to see complete or substantial change to their own jobs. Four in ten respondents fear it will have a negative effect on the structure of the workforce sooner – within the next three years.

In most occupations, however, including trading, humans are unlikely to fade from the scene anytime soon. According to Mr Hodjat, the individual trader’s role is going to diminish somewhat, but not entirely. He points out that certain types of trading expertise cannot be displaced, and that talented professionals will be needed, for example, to set up and validate the algorithms. This may frustrate the predictions of one Microsoft executive, who claimed in 2014 that, “robots will be running the City within 10 years, rendering investment bankers, analysts and even quants redundant.”

Mr Amen also believes that human intuition and influence is crucial to guide successful AI applications: “You still need to use a modicum of market understanding and intuition when you use machine learning. It’s not the case that you just put in a system and leave it for ten years; you constantly want to be coming up with new ideas which are correlated as the market changes, and that still requires humans at the end of the day.”
Mr Ebner likens the portfolio manager’s role in the age of machine learning to that of an airline pilot. “There’s structure around us and we may be on autopilot most of the way, but we enter the details into the navigation system and we decide when to engage the autopilot and when to fly manually. We’re in control of the plane.”