Monday, September 1, 2008

Fooled By Randomness

I am back.

I have just finished reading a book that has motivated me sufficiently to want to write a post: "Fooled by Randomness - The Hidden Role of Chance in Life and in the Markets" by Nissim Nicholas Taleb (2001). Taleb subsequently wrote a second book, "The Black Swan"(2007), the title of which has become popular since the recent credit crisis in America, to describe rare, difficult-to-predict events that have a large impact. Taleb was an options trader in New York, and while both books dwell a lot on financial markets, they should be interesting to anyone who is interested in epistemology, understanding the nature of knowledge.

Let's start with "The Black Swan". Till the 17th century, people in Europe 'knew' that all swans were white. That would have been a reasonable assumption to make considering that all the swans that people had seen, and their parents and grandparents before them had seen, were all white. This is an instance of inductive reasoning, where people drew a universal conclusion (all swans are white) from a set of observations (all the swans that I have seen are white). But what if someone, somewhere, saw a black swan? Everyone would have to change their ideas about swans and their color. This did happen. The early European explorers spotted black swans (Cygnus atratus) in Australia, in a place that had not been 'discovered' till the 17th century.The colour of swans is perhaps not a significant matter for most people, and I suspect that the discovery of the Black Australian Swan did not monopolize newspaper headlines in Europe for too long. But weak inductive reasoning abounds in the modern world in matters that have far greater significance to our lives.

What about this: you are approached by a mutual fund that solicits your money. The fund is run by a fund manager who has beaten his benchmark, usually some kind of index like the Sensex or the Dow, for the last five years. Would you invest your money with him? Are five observations (the fund manager's performance for each of the last five years), sufficient for you to reach the conclusion that he will grow your money in the future? What if the guy is just a lucky fool?

What if your fund manager is just a survivor? This is how it can happen. Imagine that your fund manager was one of 10,000 fund managers who started out 5 years ago. Assume that managers suceed purely because of luck, and the odds of being lucky in any given year are 50%. At the end of the first year, 5,000 managers would beat the index, while the remaining would get fired. In the second year,the number of surviving guys from that first batch would be down to 2,500, and 1,250, 625, 312 in the each of the remaining years. Your fund manager, with his 5 year streak of winning is one of those 312. He is considered a success in his profession, draws a huge salary and comes on CNBC several times a week voicing his opinion about the direction of the market. Everyone believes that he can tell a good stock from a bad one, including probably himself, till one day, his luck runs out, and he gets fired like the others before him. You lose your money.

There is evidence that this may actually be true. The New York Times had an article, titled appropriately, "The Prescient are Few", which discussed a study called “False Discoveries in Mutual Fund Performance: Measuring Luck in Estimating Alphas” by three researchers who looked the fund performance between 1975 and 2006 using a statistical test called the False Discovery Rate that eliminates the possibility that a conclusion (for e.g. skill matters in success as a Mutual Fund manager) is statistically significant when it is actually random, and the reverse. The conclusion: "the number of funds that have beaten the market over their entire histories is so small that the False Discovery Rate test can’t eliminate the possibility that the few that did were merely false positives — just lucky, in other words. Very few fund managers actually showed true stock picking ability over a long period of time." You can download the original research paper here.

What applies to fund managers applies to other actors in this world. People who are considered successful by conventional standards -wealth and fame- are very often are under the illusion, (and spread this illusion) that they are successful because they were thriftier, or smarter, or more hardworking than the rest of the population. Do all hardworking, thrifty, smart people achieve an equal measure of success? Perhaps not.

I looked up the web, and found some interesting data for the chances for commercial success for an inventor. Bob Shaver is a patent attorney in the United States who noticed that only 5 in 100 inventions ended in some sort of commercial success. Most people, including me, would think inventors are both hardworking and smart.

It may be more than political correctness, therefore, to address the poor as the "less fortunate".

As you can see, by the length of this post, I liked this book very much.

Friday, May 2, 2008

Saying hello with pride


I spotted this giant hoarding advertising a new prepaid plan from Airtel, India's largest mobile company, near my home in Bangalore. The model, is of course, Shahrukh Khan, Bollywood's leading actor. The text reads, "Garv se bolo, hello!", which is Hindi for "Say it with pride - hello!".

The last time I heard a similar exhortation was in the early nineties, when India was in the grip of a particularly vicious period of violence between its two main religious groups, the Hindus and the Muslims. The cause was a 16th century mosque, the Babri Masjid, located in Ayodhya in Northern India. The Hindus claimed that Ram, one of the most popular gods in the Indian pantheon, was born at the exact location where the mosque stood. They wanted to demolish the mosque, and build a temple for Ram. The Muslims, naturally, were very unhappy about this.

Leading the campaign for building the Ram temple, was the Bharatiya Janata Party (BJP), which is now one of the two leading political parties in India. One of the slogans they made popular during that period was "Garv se kaho, hum Hindu hain!", which is Hindi for "Say it with pride, I am a Hindu!". Now, sixteen years later, there is Shahrukh Khan, a Muslim, with his hand on his chest, demanding that we show pride when we say hello. If you look very closely at the photograph, you can even see a halo around his head. Very droll.

The other slogan that was very popular in those days with BJP supporters was "Jai Shri Ram", or "Victory to Lord Ram". The BJP asked its supporters to use this as a greeting, replacing other more widely used greetings such as "Namaste" ("I bow to you"), and even that universal greeting for telephone conversations, "Hello!".

In 1995, I had just started working as a trainee at an automobile manufacturer in Pune, when I received a phone call from a colleague who worked in another department. The conversation went something like this:

Me:Hello!
Colleague: Jai Shri Ram!

Me:Yes, thanks!
Colleague: Jai Shri Ram!

Me:Jai Shri Ram.

He was very friendly after that, having managed to get someone with an obviously Christian name to root for Ram's victory against the Muslims. I mentioned this to my other colleagues, all fairly conservative Hindus, and they shrugged it off, "oh him, he is a harmless nut" .

But other nuts were not so harmless. The "Jai Shri Ram" warcry accompanied BJP supporters as they demolished the Babri Masjid and slaughtered thousands of Muslims in the riots that followed. More recently, in the 2002 Gujarat Genocide, Muslims were asked to say "Jai Shri Ram" before they were murdered by Hindu mobs. To be fair, when the Muslims did manage to retaliate, their mobs proclaimed the greatness of Allah (Allahu Akbar).

Perhaps Shahrukh is right. There may be a lot of pride in saying just "Hello."

Links:




Wednesday, April 30, 2008

This really made me laugh.

"Sometimes, a joke will make a person laugh. Sometimes a funny picture or video will. Sometimes a clever song, a dramatic accident, the misfortune of others…nothing is ever guaranteed to make everyone laugh. Except for the sound of babies laughing."



See the other 6 videos at Say No To Crack. Each one of them is very, very funny.

Saturday, April 26, 2008

Something useful, finally.

I just got some serious dental work done- a root canal. Like most people, a visit to the dentist is not something I look forward to, so I procrastinated until what could have been a simple filling needed the distressing blasting and excavation that is a root canal procedure.

One of the ways I manage to put off tasks that I would rather not do, is to cleverly channel my energies on a 'let's-gather-all-the-information-we-can-on-this' effort. This is a ploy from my management days, but it works best on me. So while my maxillary first premolar suffered its way to becoming a candidate for a root canal, I gathered a whole lot of information on the internet about teeth. You will now understand why this post is precious.

Here is some of the stuff I learnt:

The Basic stuff: Human beings, like all mammals, are diphyodont, meaning that they develop two sets of teeth (milk/baby teeth, and the permanent teeth that come around the age of seven). Sharks are blessed; they grow teeth every two weeks (polyphyodont), though all that teething must make them very irritable. A naughty thought: if I were a shark, and in a dentist's chair, it would be the dentist's turn to be very, very afraid, wouldn't it?
Also, while being a 'dog psychologist' could net you upto half a million dollars a year, you are likely to be very poor as a dog dentist - dogs' don't get cavities because their saliva is alkaline.

The Geeky Stuff: Dentists use three different systems (notations) for identifying teeth .
  • The British use the Palmer notation, which was actually devised by a Hungarian called Adolf Zsigmondy (trust the Brits to steal the credit), who unsurprisingly, was also a politician. It is the easiest to understand, but is a pain to type because it uses some symbols which you won't find in Microsoft Word. I guess you won't find many British dentists blogging about their work.
  • The Americans have a 'Universal Number System' which no one else follows, but must go well with their gallons, miles, pounds, and 110 volt/60 Hz appliances.
  • The rest of the world follows the FDI World Dental Federation notation, which is very like the Palmer notation, but without the symbols. FDI, incidentally stands for 'Fédération Dentaire Internationale'; it is amazing how the French manage to insinuate themselves into the standards business - the metric system is also called the 'SI' system (for 'Système International d'Unités').
My tooth that got into trouble would have been '4/' in the Palmer system ( the slash being a crude approximation to the actual symbol), '12' in the American system, and '24' in the FDI notation.

The Historical Stuff: The outermost surface of teeth is composed of enamel, which is the hardest substance of the body. But obviously not hard enough to keep dentists out. Our diets are to blame, as usual. Archeologists have found a huge increase in tooth decay from around the Neolithic period (about 8500 BC) when most humans changed from being hunter-gatherers to 'settled' agriculturists, eating a lot of grain. There was apparently very little tooth decay before that, when the whole world was on the Atkins diet.

Incidentally, with the 'Neolithic Revolution' and dental decay and dentists, also came food surpluses, and kings who 'appropriated' those surpluses so that they could wage war against other kings - till Marx, and Lenin and Mao took them back for us. So that they could wage war on us. Religion came too, and the Pope, and Osama, and the War on Terror. It took John Lennon to imagine how different things could have been. But all that in a seperate post.

Something current: In 2006, nearly half a million Americans traveled abroad for medical treatment, according to this article in the New York Times, of whom about 40% are estimated to have been dental tourists. India, is apparently a very popular destination, 'putting smiles on many faces', as the Economic Times puts it.


This dentist prefers euros over dollars...




Photos are thanks to Matt Logelin who also writes an extremely moving blog here. By sheer coincidence, Vinoo sent me a Facebook message with Matt's blog just this morning. Later, while I was writing this, I came across these pictures by Matt via Google Images, and selected them, without realizing they were his.
No, the guy in the picture is not my dentist.

Thursday, April 17, 2008

Secrets on a Postcard


This secret website:
how I miss
the churches of my youth!






Bad haiku? Beautiful blog


Wednesday, April 16, 2008

Periodicity

In the early nineties, when CERN first threw the World Wide Web open for everyone, some of the happiest people on the planet must have been chemists. They now had a neat little way to represent their beloved Period Table (PT). With hyperlinks, they could finally display all the important data about each element without being constrained by those little boxes. So, they set about populating the web with hundreds of these tables, improving them diligently as the technology improved.

Some of these online periodic tables are beautiful. See this Interactive Periodic Table, by Michael Dayah, which is clearly the best functionally. Read the 'about' page to understand how to work it. Or this more artsy one, from the Royal Society of Chemistry. Both came to me via Craig Stoltz's blog, 'Web 2.0h...really?'. I thought just these two exciting enough to warrant a post.

And then, I spoke to my friend Vinoo Samuel, and he explained the weakness chemists have for online periodic tables. I searched, and I found PTs in 'spectroscopic' colors and in spiral and circular shapes. Also, some really bizarre ones, with their elements linked to comic books where they were mentioned (Krypton - Superman - Krypton Chronicles,Volume 1, Number 1, September, 1981, cover), or to science fiction stories (the first deliveries of asteroidal cobalt were flown down to the north central Pacific Ocean in the form of lifting bodies in the year 2116...). And of course, there are the spoofs, such as this 'Creationist/State of Kansas' PT.

Offline, too, there has been some truly amazing efforts to ensure that the Periodic Table is never out of sight. Theodore Gray could never make the mistake that Time.com made (an index without an index, see my previous post). As he observes with some indignation, "In the well over a century since its invention by Dr. Mendeleev, the world has consistently failed to notice the word "Table" clearly contained in the name "Periodic Table", and has insisted in printing it on paper, hanging it on the wall, putting it in the back of chemistry textbooks, and generally doing all kinds of things with it that having nothing to do with being a table.". So of course, he created a Periodic Table that was - a real table! Chemists, I assume, were also behind this PT inspired song by Tom Lehrer.

And yes, there are the dangers of overdoing this stuff, especially if the people involved are not really chemists, as this complaint about an error riddled Periodic Table shower curtain proves.

Mendeleev would be proud.


Thanks to Ron Rinehart for leading me to most of these tables.



Tuesday, April 15, 2008

Time.com's First Annual Blog Index

Time.com has published an Index of The Top 25 Blogs. I found a lot of blogs there that I had never heard of, and which I have now subscribed to. If you have as much time on your hands as I do, have a look.

But this being Time, the Index of The Top 25 Blogs does not have an index! You must click through 25 pages (each page carrying a writeup on one blog). Fortunately, the good folks at Valleywag.com (inexplicably left out of this list) have compiled a one page version which you can read here. Vallewag's article is very appropriately tagged "Blogging for Dollars", since at least here, Time.com, is guilty of a 'crass campaign to generate more pageviews'. As Fark.com's Drew Curtis, quoted in the same Valleywag article, eloquently puts it, "That's the point — pick shit people don't agree with, generate controversy, SPREAD THE FUCKING THING OVER 50 PAGES WITH NO INDEX, profit."

Or maybe, Time thinks that their readers miss the experience of turning pages in a magazine, so they gave them "Next" buttons? Naah...

If you still want to read the original article, it's here.

But of course, at the Valleywag page, you cannot tell what each blog in the list is about. If you were reading the list in a magazine, that would be a problem. But if, like me, you are reading it on the internet, and you have a decent browser like Firefox, you could right click each name, open each blog in a new tab, and see for yourself what the fuss is about.