The current state of AI for investment management

When OpenAI released ChatGPT in November 2022 it created a wave of excitement around artificial intelligence (AI) among the general public, media, and across industries worldwide. Such generative AI models (also called generative large language models or LLM) have sparked a new race in the tech world regarding who has the best AI offering. On one hand, the Microsoft-OpenAI partnership was leading the way with incorporating generative AI into many applications, while other big firms like Google were stepping up with their own AI tech in the form of BARD. More recently, open-source versions of generative AI such as Meta’s Llama are surging in popularity, because they enable end-users to finetune generative AIs for their own specific applications without the massive cost and data required to train a generative AI from scratch.

The excitement around generative AI is rooted in the vast opportunities they present to transform various aspects of our personal and professional spheres. Not only is generative AI seen as a new companion for casual conversations, but also as a versatile tool capable of authoring poems, offering legal opinions, brainstorming marketing strategies, and even selecting the best investment stocks (Lopez-Lira and Tang, 2023). Like in other industries, finance practitioners are fascinated by the possibilities. While some finance practitioners are pioneers in the use of AI, the vast majority of the industry is either in the early stages of exploring its potential or has recently begun integrating it into their workflows.

In this article, we touch on salient advances in artificial intelligence and its applications in investment management, highlighting the opportunities it offers across a broad spectrum of use cases. For example, AI can be used to uncover nonobvious relationships in massive quantities of data or to detect sentiments expressed in textual or even audio data. We start with a discussion on the key developments in AI over the years and highlight how the finance industry have been incorporating these advances. We then discuss three specific sub-branches of AI: machine learning (ML), natural language processing (NLP), and generative AI.

 

A BRIEF HISTORY OF AI AND THE CURRENT LANDSCAPE

Researchers have relied on fundamental principles and mathematical derivations throughout history to make predictions and decisions. But the increased complexity of tasks today calls for new methods and algorithms to identify more intricate relationships between many variables. To this end, more advanced systems and algorithms have been developed to conduct heavy data-processing tasks and make human-like intelligent decisions. The umbrella name for all such systems is artificial intelligence, the ultimate goal of which is a system indistinguishable from human intelligence, the so-called artificial general intelligence (AGI).

While early AI systems were unable to learn and adapt as new data came in, in time they flourished, developing into various subfields such as machine learning (ML) and natural language processing (NLP) (see Figure 1). For instance, in ML, problems are solved using algorithms that can adapt, learn, and discover their own rules and relationships, while in NLP computers are given the ability to quantify and comprehend textual and/or speech data.

To flexibly learn data patterns and nuances, ML and NLP algorithms often use deep learning models which are neural networks with multiple layers. One of the most prominent recent innovations in artificial intelligence are generative AI models, wherein deep neural networks learn the nature of the input training data to then generate new data that has similar characteristics. Well-known examples of generative AI models include GPT4 for textual applications and DALL-E for image applications.

In field after field, artificial intelligence techniques have made their way into various tasks such as facial recognition, internet searches, medicine, autonomous driving, or playing chess and Go. In this last example and indeed many other fields, machines have become fully autonomous and outperformed their human contenders.

 

MACHINE LEARNING IN FINANCE

While ML excels at many complex tasks, investment brings a whole new level of difficulty because it relates to predictions and decisions on the economy and financial markets. These are influenced by millions of human beings who are in turn influenced by a myriad of variables. The finance industry began adopting elements of AI as early as the 1990s, with varying degrees of success. For instance, consumer credit default probability and fraud detection models were among the first to use ML techniques. Conversely, the practical use of ML for stock selection and market timing beyond simple quantitative algorithms has only taken off in the last decade.

There are at least three reasons why investment management applications have adopted such techniques only recently. First, ML models require a massive amount of training data, which is more abundant in some fields than others. For example, two computerized chess players can play endlessly against each other, thus generating unlimited training data.1 The amount of data in investment applications tends to be more limited. Take the example of predicting asset returns during different macroeconomic periods. The most commonly used dataset in asset pricing, the US Stock Database of CRSP,2 has less than one hundred years of sample data and only covers US companies and sixteen recession periods (as defined by the NBER3). Predicting recessions based on sixteen unique observations is inherently more challenging than predicting the best next move in chess.

Secondly, finance is dynamic and adaptive unlike fields bound by rigid rules. For instance, pawns can only move forward in chess, but in investing, market participants have a larger opportunity set but less information. Unlike chess or Go, where all pieces are visible, investors are typically unaware what other market participants hold. This opacity, coupled with the diverse reactions of investors to similar decisions at different times, adds complexity. Investors are heterogeneous. Moreover, the unpredictability of events like the sudden abandoning of the gold standard in the US or the COVID-19 pandemic make it hard for ML to add value in investment applications.

Thirdly, stock price patterns are not obvious. Otherwise, they would be quickly arbitraged away by smart market participants. Because financial asset returns result from aggregate inputs of thousands of investors each incorporating a plethora of considerations, from macro to micro and from sentiment to positioning and liquidity needs, they are more driven by human behavior, biases, and heuristics. Put another way, the signal-tonoise ratio in finance is famously low compared to other fields in which ML has excelled as a result.

Nevertheless, there are many instances where machines can help humans in investment decisions, owing to the observation that “history never repeats itself, but it often rhymes.” While ML models do not have domain knowledge per se, the practitioners developing these algorithms do. As a result, ML models can be directed to learn the relevant features by going through many examples in the training data. These algorithms generally become more robust and accurate as machines are fed with more data, ideally arriving with a higher signal-to-noise ratio.

This highlights the benefits and shortcomings of ML models. On the one hand, while humans might be swayed by behavioral biases, machines can see the data as it is. Take the unduly high valuation of internet companies during the dot-com bubble as an example: while a human investor might have overreacted to the bright future of the internet, a proper ML model would have been less likely to have rationalized these company valuations. On the other hand, when predicting an event, machines are generally mostly successful for events with similar precedence in the training set. When the event is completely new, machines are likely to struggle. For example, ML models could barely predict the stock market swings in 2020, as they had not seen a widespread global pandemic before.

Hence, although a machine can execute a task faster than a human and bring sensible second opinions, in finance, it is not about ‘man versus machine’ . Instead, “AI power and human wisdom are complementary in generating accurate forecasts and mitigating extreme errors, portraying a future of ‘Man + Machine’ (Cao et al, 2021). Machine-learned patterns are useful only if they can be rationalized by economic intuition, something that is arrived at by human judgment.

 

INVESTMENT MANAGEMENT APPLICATIONS OF MACHINE LEARNING

In this section, we illustrate a few concrete examples of how machine learning can benefit the practice of investment management, focusing on risk and return prediction. Over the past few years, academic research has investigated the efficacy of ML in stock price prediction (see, for example, Gu et al. 2020, Hanauer and Kalsbach 2023) or bond price prediction (see, for example, Bianchi et al. 2021). Machine learning models can identify complex patterns that might easily go unnoticed by the human eye, especially regarding asymmetries and interaction effects.

Short-term price reversal: A robust pattern in stock prices is the shortterm price reversal effect, which was discovered in the 1980s. When stocks experience a large negative price move over a short period (a couple of days), this price move tends to partially reverse over the next few days. Indeed, short-term price reversal is often the most prominent feature in ML models that predict the cross-section of equity returns, see Gu et al. (2020), Leung et al. (2021), or Blitz et al. (2023a). Furthermore, ML models often find that short-term price reversal interacts with other features.

For instance, short-term reversal is much weaker after news or earnings announcements are released. Such machine-learned patterns can be economically rationalized, as price reversal happens mostly after a large market participant liquidates a large number of shares and moves the firm’s stock price away from its fair value (abstracting from any change to the firm’s fundamentals). However, when a big price movement is driven by news or earnings announcements , the price does not revert because the fundamental stock value has shifted to a new level.5 Based on Chen’s (2023) analysis, Figure 2 depicts how this interaction of returns and number of news items in the previous ten days can be leveraged to forecast future stock returns.

Distress prediction: While the ML prediction of stock prices suffers from a low signal-to-noise ratio, this ratio is more benign when it comes to risk and distress prediction. For example, we intuitively assume that increasing the cost of debt can have a negative impact on a stock’s riskiness. However, the ML model of Swinkels and Hoogteijling (2022) reveals that this effect is not linear and homogenous; increasing cost of debt is especially detrimental for heavily indebted companies with shorter distance-to-default. Overall, the authors demonstrate how modelling such non-linear patterns together with interaction effects can benefit the prediction of stock price crash risk. Figure 3, as presented in Swinkels and Hoogteijling’s (2022) analysis, demonstrates that the prediction of stock returns involves non linearity and interaction effects, for example with respect to distance to default and interest expenses.

Volatility surfaces: As machine learning leaps forward in other fields, its financial applications follow suit. For example, image recognition is an everyday use case of machine learning, wherein a machine is tasked to identify the main objects in an image. The first layers of the neural network that are used in these models are often convolutional ones that identify the primitive shapes in the image, such as lines, circles, and squares. In the later layers, these primitive shapes are combined to construct more complex patterns, such as a human portrait, with a circle for the face, two small circles for the eyes, two lines for the eyebrows, and so on. While this use case might seem farfetched, it can readily be translated into a finance application. Kelly et al. (2023) show how the volatility surface can be seen as an image, with pixel colors as the implied volatility of the options at different moneyness levels and time-to-maturity (Figure 4), and then use convolutional layers to predict stock returns with the data embedded in this image. In this setup, the machine learns the relationship between the option-implied information and the future returns of the corresponding security

Company clustering and theme identification: All of the above are examples of supervised learning, where the target (either stock return or risk) is clear. However, ML techniques are also well established for unsupervised learning, such as company clustering, where the aim is to form clusters of similar companies based on a large number of characteristics. These clusters can then be used to identify emerging market trends and themes, or for peer group comparison. Conventionally, similar companies are grouped based on their sector or industry classifications. However, these classifications are often too crude, bucketing companies in some groups without considering the closeness of companies in two distinct buckets.

For example, consider a lithium mining company and an electric vehicle producer. Although such companies are classified in two distinct sectors (mining and consumer discretionary), they have exposures to similar risk factors. In order to improve stock peer groups in this way, one can use Principal Component Analysis (PCA) or Uniform Manifold Approximation and Projection (UMAP) to reduce the data dimensionality, and subsequently use clustering techniques such as K-Nearest-Neighbors (KNN) or Density-based Clustering (DBSCAN) algorithms to group the companies based on their similarity. Figure 5 displays clusters of companies based on the similarity of company descriptions.

Trading: Once the suitable stock has been identified, it becomes crucial to employ the most effective trading strategy. For example, revealing the intent to trade a substantial volume of securities can inadvertently trigger significant price fluctuations, often detrimental to one’s objectives. Thus, traders or brokers frequently utilize advanced machine learning models to ascertain the most beneficial trading approach, such as determining the optimal participation rate, the best trading venue, etc.

The availability of more independent observations at intraday frequencies allows machines to be furnished with a larger volume of data. This, in turn, enhances their ability to identify robust patterns in the data, rendering trading as one of the most promising domains for the application of AI in finance.6 However, the application of ML in trading is not confined to optimal trade execution. It can also be extended to other significant use cases, such as broker selection. For instance, a study conducted by ter Braak and van der Schans (2023) demonstrated how reinforcement learning algorithms can be employed to identify the most effective broker execution strategies.

 

THE VIRTUE OF NATURAL LANGUAGE PROCESSING

Along with the development of ML models, natural language processing (NLP) models have made great strides in performing textual analysis. The first NLP models such as bag-of-words were statistical and only based on the coappearance and frequency of words in a text. These primitive models treat each word as an independent observation. For example, the words “car” and “vehicle” are treated independently, although these two words are similar.

These models cannot gain much understanding of the meaning and similarity of two distinct words, and they cannot provide a deep understanding of the context and the relation of different words and sentences in a document. To overcome these shortcomings, deep learning models have been developed that can embed each word within a vector of numbers. Words with more similar meanings would have a smaller cosine distance between their embedding vectors, and one can even perform simple calculations with these vectors, such as “King – Man + Woman = Queen.”

The invention of attention layers and transformer models brought another giant leap to natural language processing models, see Vaswani et al. (2017). Such models can learn the word’s context within a context window (for example, within the sentence or paragraph), providing a much deeper textual understanding through the machine. Below, we highlight how NLP models can be applied to sentiment analysis as well as help advance sustainability investing.

Sentiment Analysis: The abundance of financial textual data makes it a unique hunting ground for testing NLP models. For instance, sentiment analysis helps to gauge the tone of management speech or company news. A simple way to identify text sentiment is to measure the proportion of positive and negative words. However, this naïve approach can be improved upon by leveraging more recent NLP developments as presented in context-aware models tailored for financial texts such as FinBERT (2020).

Indeed, Heston and Sinha (2018) show that news sentiment can help predict stock returns at different horizons, and Jiang et al. (2019) show that higher manager sentiment (extracted from company 10-K fillings, 10-Q fillings, and conference call transcripts) precedes lower aggregate earnings surprises and greater aggregate investment growth.

Sustainability investing: Sustainability is a fruitful area of AI application. For instance, in the past, company ESG scores were primarily based on self-declared surveys filled by the company management. Today, company news (media perception), earnings calls, financial disclosures (management perception), or employee reviews (employees’ perception) can be rich data sources for company profiling. For example, one could score companies on ESG aspects based on how the media perceives them in each dimension. Similarly, Amel-Zadeh et al. (2021) apply NLP models on Corporate Social Responsibility (CSR) reports of Russell 1000 companies to measure their alignment with UN Sustainable Development Goals.

To perform specific sustainability investing tasks, LLMs have also been trained and fine-tuned to perform specific sustainability investing tasks. For instance, ClimateBERT, ESGBERT or ControversyBERT can aid in investigating a company’s climate disclosure to spot greenwashing or identify companies linked to controversial behavior, see Webersinke et al. (2023), Schimanski et al. (2023) and Lohre et al. (2023).

 

THE RISE OF GENERATIVE AI

The most prominent recent innovations in artificial intelligence are generative AI models, particularly GPT models in natural language processing. These models are trained on vast text datasets and can interact with the user, answer questions, edit given documents, and assist in ideation by generating sensible sentences and paragraphs. As such, they can be used in the automation of tasks, not just in finance but in many industries. They can thus reduce the burden of repetitive labor-intensive tasks, such as drafting regular reports, requests for proposals, or emails. It is important to caution that generative AI models (e.g., ChatGPT) are not flawless and require interactive supervision by a human. In other words, humans function as domain experts and validate rather than create the content.

But the use cases of GPT are not limited to repetitive tasks. With the release of generative models tuned for programming, such as Github CoPilot, one can enjoy the machine as a sparring partner in coding functions or when translating them from one programming language into another one. GPT models can assist lawyers by analyzing large volumes of legal documents, case laws, and regulations and providing a second opinion. In customer relationship management, GPT models can power chatbots to manage initial customer inquiries, provide automated responses, and offer basic troubleshooting guidance. They can understand and respond to customer queries based on pre-trained knowledge and historical data from the company.

Finally, GPT can function as a helpful research assistant in various ways. It can help generate ideas, conduct basic literature reviews, summarize academic papers, compare arguments between two papers, edit drafts, and answer basic questions like identifying peer companies of NVIDIA or determining which sectors are impacted by energy price inflation.

 

IMPLICATIONS AND CONCLUSIONS

The introduction of AI in finance is a significant advancement that can revolutionize various office tasks. While concerns about personnel redundancy and unemployment are certainly valid, history has shown that humans can adapt and find new avenues for growth and development alongside automation. For example, the rise of the steam engine replaced horse cart drivers, but created jobs such as train engineers, railroad builders, conductors, and so on.

The rise of computers replaced people that did manual computations, data entry, and record keeping, but created new jobs such as programmers, database engineers, or system architects. Similar to the way in which robots automated factory jobs in the past, AI has the potential to free up human resources to focus on higher value-added and creative activities, such as idea generation and business development. As we witness thousands of humans collaborating on NASA’s “unmanned” Artemis program to explore the moon, it becomes evident that humans and machines can work together synergistically.

More specifically for finance, the introduction of AI can automate many of the repetitive and labor-intensive processes currently carried out by human practitioners. Examples include summarization of news and broker reports for consumption by portfolio managers and writing strategy performance reviews for asset owners to keep track of their investments. Other, more intellectually involved aspects of finance may also be speeded up or completely automated by AI, such as finding unique and causal relationships within massive amounts of data or deciding in which asset class to invest, depending on macro-economic data and market sentiments.

What about disruption posed by AI? For the finance industry, on a macro level, implementing artificial intelligence will allow the industry to further evolve. Fundamental investing will experience a particularly large disruption. Until recently, AI was mainly used by advanced quantitative investors. As AI tools become more user friendly, they will allow fundamental investors to acquire the capabilities previously only available to quantitative investors. With AI, fundamental investors will be able to systematically analyze large datasets, perform highly complex calculations, and so on.

However, this does not mean quant investors’ traditional advantages will dissipate in the age of AI. Many AI programs are trained for general tasks. For finance specific applications, further trainings (called “fine tuning” in AI lingo) are necessary. Skilled quant investors will have the tools and know-how to customize general AI programs or develop completely custom AIs from the ground up. These specifically customized AI will have an advantage over the more generic AI employed by fundamental investors.

At the individual level, certain skills and jobs will disappear while others will evolve as AI rewrites the rules of the game. While certain human capabilities are not in danger of being replaced for the foreseeable future – think of empathy, creativity, and the creation of new knowledge – adopting and harnessing the possibilities of AI will be key for investment professionals to stay relevant. For today’s practitioners, it is beneficial and perhaps even essential to become comfortable in working with AI, at the very least. Learn the basics of AI – there are many useful and easily accessible courses available – and think about which parts of one’s daily work can be made better by incorporating AI into the process.

 

References

• Amel-Zadeh, A., Chen, M., Mussalli, G., & Weinberg, M. (2021). NLP for SDGs: Measuring corporate alignment with the sustainable development goals. Columbia Business School Research Paper.
• Bianchi, D., Büchner, M., & Tamoni, A. (2021). Bond risk premiums with machine learning. The Review of Financial Studies, 34(2), 1046-1089.
• Blitz, D., Hoogteijling, T., Lohre, H., & Messow, P. (2023a). How can machine learning advance quantitative asset management? Journal of Portfolio Management, Quantitative Tools 2023, 49(9), 78-95.
• Blitz, D., Hanauer, M. X., Hoogteijling, T., & Howard, C. (2023b). The Term Structure of Machine Learning Alpha. Journal of Financial Data Science. Forthcoming.
• Cao, S., Jiang, W., Wang, J. L., & Yang, B. (2021). From man vs. machine to man+ machine: The art and AI of stock analyses (No. w28800). National Bureau of Economic Research.
• Cartea, Á., Jaimungal, S., & Sánchez-Betancourt, L. (2021). Deep reinforcement learning for algorithmic trading. Available at SSRN 3812473.
• Chen, M. (2023). Eight things you should know about next-gen quant, Robeco white paper, https://www.robeco.com/en-int/ insights/2023/08/8-things-you-should-know-about-nextgen-quant
• Dai, W., Medhat, M., Novy-Marx, R., & Rizova, S. (2023). Reversals and the returns to liquidity provision (No. w30917). National Bureau of Economic Research.
• Gu, S., Kelly, B., & Xiu, D. (2020). Empirical asset pricing via machine learning. The Review of Financial Studies, 33(5), 2223-2273.
• Hanauer, M. X., & Kalsbach, T. (2023). Machine learning and the cross-section of emerging market stock returns. Emerging Markets Review, 55, 101022.
• Heston, S. L., & Sinha, N. R. (2017). News vs. sentiment: Predicting stock returns from news stories. Financial Analysts Journal, 73(3), 67-83.
• Jiang, F., Lee, J., Martin, X., & Zhou, G. (2019). Manager sentiment and stock returns. Journal of Financial Economics, 132(1), 126-149.
• Kelly, B. T., Kuznetsov, B., Malamud, S., & Xu, T. A. (2023). Deep Learning from Implied Volatility Surfaces. Swiss Finance Institute Research Paper, (23-60).
• Leung, E., Lohre, H., Mischlich, D., Sheah, Y., & Stroh, M. (2021). The promises and pitfalls of machine learning for predicting cross-sectional stock returns. Journal of Financial Data Science, 3(2), 21-50.
• Lohre, H., Nolte, S., Ranganathan, A., Rother, C., & Steiner, M. (2023). ControversyBERT: Detecting social controversies and their impact on stock returns. Journal of Impact and ESG Investing, forthcoming.
• Lopez-Lira, A., & Tang, Y. (2023). Can ChatGPT forecast stock price movements? Return predictability and large language models. arXiv preprint arXiv:2304.07619.
• Ritter, G. (2017). Machine learning for trading. Available at SSRN 3015609.
• Schimanski, T., Reding, A., Reding, N., Bingler, J., Kraus, M. and Leippold, M. (2023). Bridging the Gap in ESG Measurement: Using NLP to Quantify Environmental, Social, and Governance Communication, Working Paper.
• Swinkels, L., Hoogteijling, T. (2022). Forecasting stock crash risk with machine learning, Robeco white paper, https://www.robeco.com/en-za/insights/2022/06/ forecasting-stock-crash-risk-with-machine-learning
• ter Braak, L., & van der Schans, M. (2023). Optimal Order Routing with Reinforcement Learning. The Journal of Financial Data Science.
• Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
• Webersinke, N., Kraus, M., Bingler, J. A., & Leippold, M. (2022). ClimateBert: A pretrained language model for climate-related text, arXiv preprint arXiv:2110.12010, AAAI Fall Symposium 2022 (Conference Proceedings).
• Yang, Y., Christopher, M., & Huang, A. (2020). FinBERT: A Pretrained Language Model for Financial Communications. Available at: https://github.com/yya518/FinBERT.

Notes

1. For example, AlphaZero AI model of DeepMind for learning and playing chess, Go and Shogi.
2. Center for Research in Security Prices, LLC, affiliate of Booth School of Business at University of Chicago.
3. National Bureau of Economic Research, an American private nonprofit research organization.
4. While short-term price reversal looks strong on paper, models that predominantly rely on short-term models exhibit high turnover and the associated transaction costs may reduce the effective value-add in actual investment portfolios, see Leung et al. (2021). Blitz et al. (2023b) further document that short-term price reversal is particularly important when the forecast horizon is short.
5. For more discussions on reversal and its interactions, see Dai et al. (2023).
6. See for example Ritter (2017) and Cartea, Jaimungal, Sánchez-Betancourt (2021).

 

15 January 2024 in VBA Journaal door Mike Chen, Iman Honarvar and Harald Lohre