Christopher Mann, MSc CEng MIET MIoP
Head of Data, Overstone Art Services
As we’ve all seen recently, Large Language Models (LLMs) are transforming not just AI but the wider world. Here at Overstone we've been thinking about how to use them in the data-driven, art-finance world. We see the huge potential of LLMs to enhance our services and revolutionise the art finance industry. To explore the possibilities, we undertook a short proof of concept study into two applications.
After discussing a range of potential applications, from co-authoring legal documents to surfacing insights from data, we settled on two use cases: enhancing the efficiency of our machine learning data pipeline and enriching our asset management platform's user experience with an art advisor chatbot.
LLMs in a Data Pipeline
For the data pipeline, we tested GPT-3.5-turbo’s ability to extract artist names from art auction listings. We have already automated this processing step with high accuracy, so we wanted to see if an LLM could do better.
We found that the GPT model was able to extract artist names from lot descriptions with an accuracy of almost 90%. Moreover, when controlling for the probabilistic nature of the models via strong context system prompts and setting the model’s temperature to 0, the outputs were almost entirely consistent, making them easy for post processing. However, there were issues, such as publishers’ names reported as artists. More importantly, our data pipeline already achieves over 99% accuracy on this task, so our proof-of-concept setup would need considerable improvement to make a switch to LLMs worthwhile.
LLMs as an Art Advisor
We used GPT-3.5-turbo again for the part. Our CEO had already tested ChatGPT by asking it some tough questions about artists and their works and the results were very impressive, so we had high expectations!
We built a chatbot that would not only recommend artists based on the public data it had been trained on but also by integrating it with proprietary data. Of critical importance here was data security and confidentiality. Hence our design had strict controls to ensure only publicly accessible data was sent to OpenAI’s servers.
The result demonstrated the feasibility of a chatbot that could highlight artists to our clients whose works they might like to buy, based not just on artistic similarity but also incorporating data on which artists are part of the most successful collections, such as those that had shown the largest growth in value over the last year. Again, there were some issues with the output of the model, so NLP techniques would need to be applied to the outputs, but the results were very promising.
In conclusion, our proof-of-concept study has demonstrated the potential application of LLMs in the art finance industry. Whilst their incorporation into data pipelines may be some way off, they have massive potential. Our chatbot experiment was highly successful, and while the work to build out an industry-leading experience based on it has just begun we are excited by what the future holds with this exciting new technology and by the experiences we will soon be able to deliver to our clients with it.
Christopher Mann has nearly a decade of experience in data science and advanced analytics, working for organisations including Rolls-Royce, BMW, Unilever, Network Rail and the Home Office. Most recently he has led a team of data scientists to develop a state-of-the-art demand forecasting capability for Deloitte.
Chris brings expertise in a wide range of areas, including frequentist and Bayesian statistics, optimisation, data mining, agile and software development. Chris holds a BSc in Physics, an MSc in Nuclear Technology (specialising in probabilistic modelling of uncertainty) and is a chartered engineer and member of the Institute of Physics and Institution of Engineering Technology.