31 March 2021
What is big data?
In our industry Big data is anything that doesn’t fit in excel. Real big data is something that doesn’t even fit in a regular computer.
- Applications to investing - there is a dust trail we all leave when browsing websites online.
- Weather data can correlate with things like sales growth.
- Big data analysis can help answer questions and fill in gaps in the understanding of the world. Have just a little more information, or interpret it better.
- Fund managers are natural “weighers” of lots of different data points. It is natural to work in terms of confidence levels.
- The key is to master where data is useful and where it isn’t. Sometimes the trend is useful, but the level isn’t. You can’t wait until its perfect and solid as speed vs solidity trade off is at work in the investment industry.
- There is so much you can do with data that is clever but not useful.
- So the first question you have to ask is – is it useful? Is it something that can be acted on, and does it already exist or not?
- Focus on answering questions. Make it easy as possible to log questions – the questions themselves tell you something interesting.
- The act of trying to answer a question is interesting – often the data is too biased, too messy or too partial. So it doesn’t work.
- Focus on insights – an insight is something that changes your understanding of the world. Core to this is what the existing knowledge/understanding is.
- Data Insights is NOT quant investing.
- Data insights is about filling blindspots. Things you don’t know are missing, but if you were to fill it in you see things you wouldn’t otherwise see. That can be a competitive advantage in asset management.
- Geospacial questions is another interesting one – for example space to grow and open more stores.
- The sell side also have data science groups and do this sort of work more and more.
- Some datasets and techniques are available to anyone – webscraping is one example.
- Other datasets are vastly expensive to access and store – eg mobile phone location data.
- The “alternative data” industry is growing – sell you access to a terminal to interrogate these data sets. But the issue is you’re accessing the same info as everyone else.
- Data science is a real team sport – very diverse set of skills and expertise needed to set up a rounded data insights team.
- Things have changed in terms of hardware: Hadoop was big about 10 years ago which took a set of computers and turned it into a supercomputer at much lower cost. This was behind the growth of data. Now this is being replaced more by the cloud. Snowflake as a particular database technology is very interesting. There is a Pay as you go model emerging for data.
One thing to take away
The value of data comes from the insights – it’s about connecting data with people making real decisions. It’s a team sport to find the insights.
Thoughtful, simple analysis can get you quite far. People get carried away with “big data” and AI but things like a well-designed survey can give great insight.
How to access our podcasts
You can access all our podcasts across all major audio platforms via computer or devices.
Each LCP investment uncut podcast is for information and marketing purposes only and does not constitute any form of investment or financial advice or a financial promotion (under the Financial Services and Markets Act 2000). All views expressed by the podcast hosts and guests are purely their own opinions and do not represent those of LCP, its clients or affiliates. Our podcast listeners should always seek independent financial or legal advice before making any financial or investment decisions. Please refer to the Legal Notices section on the LCP website for further information.