“The new Chief Data Scientists should be a mix of Chekhov, Edgar A Poe, Arthur Conan Doyle and Chesterton”
Due to different vital circumstances, until recently I could not see chapters of the Black Mirror series, but one really impressed me (not the one about the pig). This is the chapter called Nosedive, where society works through ratings or social scoring (I advance that this is not so dystopian, in China the so called social credit system has already worked for a few years).
In Nosedive, each of the social interactions that occur, individuals, with their mobile, can assign from 0 to 5 stars (5 being the highest rating). If our main character (role superbly played by Bryce D. Howard) considers that a waiter has treated her well and made her a good coffee, she will assign him 5 stars. If at work a colleague is rude, they will be given a 1 star and so on. Any photo uploaded to the network is subject to qualification by the rest.
It is a society that is truly enslaved by its social rating, which for example would allow (if it is higher than 4.5) to rent a prime apartment, or, if the score falls below 3.5, to kick you off work. In this context, our protagonist wants to rent an apartment within a prime community, which asks for a minimum 4.5, but she is at 3.8. Then, advised by a consultant who is fluent in social networks and how to improve that scoring, she looks on the networks that an old friend from her school, who she had not seen for 20 years, has a score of 4.5. She texts her, and her friend quickly answers, and even invites her to her wedding to make a speech, full of people with a rating higher than 4.5.
It is the opportunity for our protagonist to level up, but after several misfortunes before the wedding, her score drops to 1.5. Her friend, upon noticing that drop, prevents her from entering the wedding and acknowledges that she had invited her because the act of inviting an old friend with a 3.5 would raise her score (as an act of charity, I imagine). I’m not going to do a spoiler, although many readers will sure have seen it, because as I mentioned at the beginning, I am late with Black Mirror.
What matters to me here is the internal dynamics of the scoring, and what are the levers that make it go up or down. A priori, if someone A with a higher score rates someone B with a lower score, it should raise the score for B, but if someone with a lower score B rates (well or badly) another with a higher score A, should that make the score go up or down for A? All this depends on the treatment of the data and the model used to predict those values. But above all, the analysis has to do with the social consequences that this fact (a person with lower score gives 5 stars to someone with better score) has historically produced … well, we are talking here about anthropological and cultural issues, but those of us who once worked and still works with data (and in trying to extract insights from that data) call them “the devil is in the details.”
In some contexts, when data abounds, it is easy to assign scores, but what is proving hard is gaining insights from them. When we buy on Amazon we can rate the purchased product. The famous “reviews” are a very important claim for the seller: products with 5 stars sell more than those with 4, and so on. But recently an investigation by the Financial Times revealed that of the 10 most active reviewers, 9 were “fake”, that is, they were paid by brands to assign 5 stars to all their products. It was easy to discover the scam: Amazon labels a review of a legitimately purchased product as “verified purchase” (to give a strong truth to that opinion), but to overcome this, companies (most Chinese) agree that the fake reviewer buys the product, and immediately sends a transfer of equivalent money via Paypal; that way the product is bought on Amazon and labelled as “verified purchase”, but the fake reviewer does not pay anything (indeed, the Financial Times discovered the scam because the products received by the fake user were then automatically sold on Ebay as brand new).
In Bezonomics, a highly recommended book for those interested in understanding Amazon’s “flywheel” business model, there is a chapter dedicated to fake reviews: the problem has already been identified by a former Goldman Sachs banker, Saoud Kalifah, who created a startup called Fakespot, which designs algorithms that are used to discriminate between true recommendations from false ones. Fakespot calculates that 30% of opinions in Amazon are fake. His advice: if you are interested in a product, the most truthful opinions are usually those with 3 stars, those with 5 stars are usually a lie, while those with 1 usually refer to the product was damaged, or was of a different colour other than the original order.
In fact, Amazon keeps sending me recommendations for products that I have already bought, and for books that I don’t like, Netflix recommends movies that I don’t like and instead, it doesn’t recommend others that, after intense manual searching, I find and I like them (the problem of false positives) and finally, LinkedIn allows me to continue receiving messages from people I don’t know and want to sell me magic software solutions, or round tables where I don’t want to participate , despite having unsubscribed in 2014 precisely because of this fact (when I unsubscribed, they contacted me from the marketing service to find out the causes, and I said that I felt like a LinkedIn product for sale, and suggested they paid me to be in, but they refused). I returned to LinkedIn recently, because I like to have 2nd chances, and that’s why I give them. As we can see, in all large , data intensive companies that use big data and artificial intelligence, the devil is in the details.
Despite the fact that there are a lot of clad beret-clad hipster gurus in San Francisco Bay who spend their days singing the benefits of big data and artificial intelligence, I have never seen them talk about the B-side of big data, probably because if they did, they would not be able to get those six figures salaries: big data eliminates causality (we do not know “the why” of things) and yet enhances correlation (the “what” of things). In addition, we have been sold a false sense of security: if there is an artificial intelligence machine doing the job, the machines are perfect. But the devil is in the details.
In 2002 I started working in a consultancy firm, because I liked to combine econometrics and banking. At that time, Basel 2 allowed us to learn a lot from this, since it forced banks to develop loan expected -loss models, and thus allocate impairment provision based on the real risk profile of each borrower. We run logistic regression models: from financial and non-financial variables of a company (balance sheet and P/L ratios, behavioral ratios based on current accounts, whether the company had a modern factory or not, whether the management had studies, etc.) we could predict the behavior of a dichotomous variable 0/1 (the company has not defaulted or has defaulted on the loan).
Soon I realised two things: junior consultants, who had to work with data in a massive way, were given the simplest laptops which caused problems when dealing with large masses of data, while the partners (not skilled in number crunching nor in interpreting data) used the more powerful computer, when its only purpose was, I suppose, to send emails to meet customers for lunch or to play golf.
The other, more important thing that I realised is that the human being is essential when dealing with data: I worked in a team with three statisticians (today we call them “data scientist”). His job consisted of building the databases, data- cleansing (it takes 80% of the work, as an skilled number cruncher would attest) and running the model. My job was to get insights from them. One day they gave me the output of a model that predicted defaults in a portfolio of loans to SMEs, and I realised that the debt / equity ratio behaved inversely to what was expected: companies with a higher level of debt (i.e., highly leveraged) defaulted less (that is, they were less likely to default than those that were less leveraged) … that didn’t make economic sense.
The scientists racked their brains looking for errors in the model, testing adjustments, trying to explain the reason for this behaviour with sophisticated statistical tests, until I asked them to see the excel sheet that contained the database: after a quick visual inspection, I jumped to the fact that the 1/0 column of the dichotomous variable contained half the empty values: simply due to some error in the data treatment, that column was wrongly copied and although it passed through the hands of three scientists, none of them noticed. The devil, again, is in the details. And this fact, in all different shapes and situations has been repeated throughout my professional life: for this reason, like air plane pilots, who carry out an exhaustive check list of the operation of the device before each take-off (link to an interesting article on the history of the check list in aviation), I have my own data verification checklist when I receive results of an analysis, a check list that increases proportionally to the number of data scientists who have previously participated in data preparation.
Large disasters in the use of data, such as plane crashes, are not caused by single and complex events, but by a succession of small, sometimes insignificant, errors that accumulate and end up causing disaster. Malcom Gladwell in his book Outliers explains that major plane accidents, after many post-accident reviews, occur after 7 errors, each of which, individually, are not important (little motivation of the crew, distractions or interruptions, desire to quickly finish the check list, etc). It is a lot less frequent, nearly neglible, that a turbine detaches from the fuselage in mid-flight.
In banking, for example, the case of Banco Sabadell with its English subsidiary TSB is well known. In 2018, 1.9 million customers were unable to access their online accounts for days. According to The Guardian, the problem was due to TSB executives not asking the right questions during the course of a platform migration project and agreeing to the implementation of the new platform knowing that the project had certain work streams as much as seven months behind schedule.
In short, executives at TSB did not have a realistic picture of the project. It does not seem like a technical, software code issue, nor of a weird algorithm that nobody understands were causing the glitch … but pure common sense.
And I think it is just common sense that can play the role of the murderer of the “devil that is in the details.” And to have common sense there is no magic recipe, but I give you a clue.
In the book “Sensemaking: the power of humanities in the age of algorithm”, the author Cristian Madsjberg advocates that the best CEO can read an Excel sheet, but also a novel. It is not about being a Luddite or not understanding the value of the data generated by algorithms, but about going one step further and realising that a human mind molded on readings of classics, novels and the Humanities in general, can solve problems that are not within the reach of a soulless or emotionless computer.
The companies best positioned to be “customer centric” will not only need the use of big data and artificial intelligence to better understand their customers, but will also require the use of thick data, that is, knowledge of culture, history, and structures underlying human behavior.
Reading novels by classical authors will help us to train our capacity for critical analysis and therefore common sense, which is highly critical and valuable when trying to find data insights.
And this is where authors like Chekhov come in: reading some of their short stories (“Failure”, “Enemies”, “The lady with the dog”) serves to make more complex analysis and question our first impression of things that, erroneously, one would think capable to understand at first sight. Chekhov directs our attention to less conventional realms of human behavior, and makes us discover details that often go unnoticed.
In Chekhov there are no predictable attitudes towards anything. If something can be described as typical in Chekhov, it is his insistence that we remain very attentive to the nuances of life, as opposed to our ordering intelligence that always urges us to order, and to clarity.
Arthur Conan Doyle, through Sherlock Holmes, teaches us that although in everyday matters it is useful to apply forward-induction (from certain facts, to come up with a result), to solve a complex problem, it is of paramount importance to reason backwards: starting from a result, extract the steps/causes that have led to it (“backward induction”). In fact, in banking there is already “reverse stress testing” which now is being implemented for Covid19: instead of imputing different macroeconomic parameters and those of the balance sheet itself to determine if the bank is sound enough, it starts from the premise that the company is bankrupt, and reasons backwards, identifying what have been the causes and values of those parameters that led to the disaster, in order to avoid it in the future.
Edgar Allan Poe (for example, in “The mystery of Marie Rogêt), trains us to be able to understand that “most of the major valuable discoveries we owe to collateral, incidental or accidental events: it has become necessary, with a view to progress , to grant the widest space to those inventions that are born by chance and completely outside of ordinary hopes ”. In fact, it was this story that helped the police uncover the actual perpetrator of Marie Rogers’ murder. Poe wrote the story using his critical analysis after reading the newspaper about this murder, piecing together seemingly unrelated facts, and going against rational thinking.
Reading Chesterton (“Autobiography”) I learned that if someone knows one thing, they are most likely right, but if someone knows that they know one thing, they are most likely wrong. I also learned that “it is not the children who should read Lewis Carroll. Rather, it is the wise gray-haired philosophers (I would add, and data scientists) who should sit down each night to read “Alice in Wonderland” to study the darkest metaphysical problems: the border between reason and unreason, the nature of the most erratic of spiritual forces, the humor that eternally dances between one and the other “
In short, we need Chekhovs, Poes, Chestertons and Conan Doyles as Chief Data Scientists at Amazon, Google, Facebook, and the big banks, monitoring and supervising the cap-clad hipsters who invite you to their Linkedin lectures while drooling about the predictive gradient boosting models.
