Randomness and Arbitrariness

Freedictionary.com defines random as something "that has no specific pattern, purpose or objective"  The same website defines arbitrary as " determined by chance, whim, or impulse, and not by necessity, reason, or principle". Broadly similar definitions, but there is a world of difference between the two (at least in my eyes).

Randomness is God (More on that in a different post, may be. Or further down in this post itself). Arbitrariness is the poor cousin of randomness exhibited by people who claim to be handling the levers of supposedly random processes. There is a world of difference. Let me give a parallel with a quote from one of my favourite books- Catch-22.

“History did not demand Yossarian's premature demise, justice could be satisfied without it, progress did not hinge upon it, victory did not depend on it. That men would die was a matter of necessity; WHICH men would die, though, was a matter of circumstance, and Yossarian was willing to be the victim of anything but circumstance. But that was war. Just about all he could find in its favor was that it paid well and liberated children from the pernicious influence of their parents.”

Somewhere in this quote is sitting the idea that I want to elaborate on. Sooner or later I will get to the point. This is a post about randomness,so the least you can expect is non-linearity (which is not even a distant cousin of randomness - which as I have mentioned before, and will/might write a post on is God).

That a service tax inspector has to check some records is a matter of necessity. WHICH companies would be verified is supposedly random, but actually based on the arbitrary selection of the service tax inspector. Who, as they do, chooses the noble path of least resistance, and zones in on two kinds of people - 1) who will give him lots of hugs and cuddles for not looking into their records and 2) will be happy to pay an arbitrarily (not randomly) higher amount of tax because they realize that debating tax norms with said service tax inspector would give them as much joy as chopping their limbs off - one at a time, very slowly with a tiny compass from a stolen geometry box. So, said government employee meets his target by being a pain and gets his bonus by dropping the pretext.

And precisely because of this combination of arbitrariness and path of least resistance, government machinery is designed to be odious. It blesses the controllers of randomness engines with powers of arbitrariness and here the seeds of corruption are sowed. But let us not belittle a discussion on randomness with thoughts on government employees.

True randomness is painfully difficult to attain. Several proxies exist, and these are pretty good, but apparently a great many of them fall short of the tests designed to test randomness. Random selections should have no bias. But most proxies aim to achieve randomness by removing bias. Bias is removed by making things equiprobable. So, many randomness generators are just bias-removers. And these two are not quiet the same.



Randomness is the key reason why most statistical inferences are actually baseless. Let me give you an example - Statistical analysis of reams of stock price data might tell you that a three-day window of investment going from Thursday - Tuesday outperforms any other three-day window. Some 'analyst' will give us his genius reasons for this as well. But truth remains that if we compared returns over a three-day window across all known stock-market data, we are very likely to find one three-day window 'outperforming' the others. So, this is just one occurrence in a random series; not an underlying trend. This distinction is something that many people often overlook (some people calling themselves analysts willfully overlook this, others are often just blindsided)Nicolas Nassim Taleb, a famous author has written a book where he focuses (almost exclusively) on this distinction.

Back to arbitrariness. Human beings are diagnosed as having a fairness gene. So, seeing something as unfair grates. A lot. Randomness is something that we can accept. It would really help if some of these random processes were selected with randomness generators in a transparent manner. I hate it that the randomness generation is left to individuals. That just leads to arbitrariness. The traffic constable catching a 22-year old girl to check the license/RC book is an arbitrary selection. If the gentleman could be made to stop every 364th vehicle and check for all this, it would be wonderful.