Enter
the Big Problem Solvers
Mathematicians are
hot hires in companies to make sense of enormous amount of data and more
Sequencing a person’s genome is an expensive and complicated exercise, but the benefits can be enormous if done well and quickly. You could calculate the risk of getting major diseases, develop new drugs, or provide targeted therapies. But because of the cost and difficulty, routine gene-sequencing has not started in a big way in India. Strand Life Sciences, a Bangalore-based life sciences company, decided recently to test the waters with a pilot project. It has sequenced the genome of about 20 people, and hopes to start a commercial service soon.
Such a service would have been unthinkable even two years ago. To sequence a genome quickly, the DNA has to be chopped up into bits and the pieces sequenced separately. Assembling these pieces of information into a meaningful whole is difficult, and requires special resources and mathematical techniques.
Instrumentation technology has advanced rapidly in recent times, but some of the most dramatic improvements have been in mathematical techniques.
And these have resulted in a precipitous drop in the cost of gene sequencing, which is now set to make a high impact on our lives. “Genome data is very different from other data,” says Ramesh Hariharan, chief technology officer, Strand Life Sciences. “But algorithms and special mathematical techniques to deal with it have improved rapidly in the last two years.”
The Plus Factor
It is a quiet revolution happening away from public view, but it is going to transform technology and business. As mathematical techniques improve, many day-today problems are being solved mathematically, thereby helping companies and governments take better decisions and forecast trends more accurately. Some of it is led by big data analytics companies, but the trend is deeper and more widespread than big data. A data-rich world is infinitely interesting to applied mathematicians, as they look for beautiful patterns and striking correlations among the huge amounts of data being collected every day. Says Bernard Meyerson, vice-president, IBM Research: “The magic of mathematics is the only way to discover useful information from so much data.”
In recent times, IBM Research has been using mathematics to investigate some uniquely Indian problems like low-cost traffic speed analysis and standardisation of addresses. Across the world, IBM employs arguably the largest pool of mathematicians in a private organisation. They look at the world in a distinctly mathematical way, breaking down conventional business problems into equations that can be solved by modern computers. IBM is representative of a larger trend in the technology world, as private companies use mathematics increasingly to solve a variety of problems from customer acquisition to predicting infant mortality.
In the last few years, IBM has invested $15 billion in companies with such capabilities. During this period, its engineers have applied this capability to a mindboggling variety of situations. They have used mathematics to predict failures in semiconductor plants, plan marketing campaigns, understand visitor behaviour on websites, manage public water supply, and spot infections in infants well before the symptoms manifest. Apart from core industrial areas, IBM also
researches topics like astrophysics,
genomics and climate change because its mathematicians have seen fascinating connections between natural phenomena and its core businesses.
Number Power
Other IT companies are building this capability through acquisitions or by hiring a large number of applied mathematicians. Business analytics is an industry built around statisticians and computer scientists, both of whom are applied mathematicians in different garbs. Internet start-ups need good mathematicians for their existence. Telecom companies need them to learn how to retain customers. Life sciences companies require them to model the body’s processes or discover whether drugs are effective or not. Financial companies, after learning valuable lessons in 2007 and 2008, are now using mathematicians to question the assumptions behind their models and develop more dependable ones for the next decade.
Oil companies use increasingly sophisticated mathematics for oil discovery and extraction. The emerging breed of complexity theorists looks at the social world in new ways, and are trying to understand collective human behaviour using mathematics. Amidst all this, a large number of IT companies now use sophisticated mathematics for their routine operations, far more than they did a decade ago. Many of the methods are not new, but mathematicians have found new ways of combining them. “Statistical models have improved considerably in the past 10 years,” says Leland Wilkinson, founder of the statistical software company Systat. “You can now get huge leverage by combining Bayes’s theorem and multivariate analysis, or graph theory and Bayesian statistics.”
Many start-ups now flaunt their mathematical capability strategically. Take the Chicago-based Mu Sigma, set up in 2004. Named after two Greek letters used in probability theory, it has raised $163 billion in the past five years, about four-fifths of it just last year. A single investment of $108 million last year was among the largest ever in an analytics services company.
Mu Sigma tackles problems like pricing of insurance policies, understanding customer loyalty to retailers, designing sales-force compensation tiveness, assessing airline ty, predicting product diffu so on. In a recent project, some of its consultants had to learn geophysics as found patterns of Himalay snow-melting matching with the business problem it was trying to solve. “Mathematics is our differentiation,” says Mu Sigma founder Dhiraj Rajaram, “and it combines art and science with scale.”
The kind of mathematics that Mu Sigma uses is now fairly established in the analytics industry: statistics and probability theory, operations research, artificial intelligence, machine learning. At the heart of many analytics companies is Bayesian statistics, derived originally from the century-old Bayes Theorem, but combined with iner areas.
ematics of human has inspired many s. Genetic algohich originated in but now present in sophisticated forms, are widely used for optimisation of resources. Optimisation is indeed a major problem tackled by many internet companies, as their businesses depend on how it optimises traffic and revenue. “Our ability to analyse big data is still limited by technology,” says Srinivasan Seshadri, former IIT professor and founder of Bangalorebased Allthingscustomized.com. “I expect this to improve in the next few years.”
Microsoft Puts
Two and Two Together
Rapid technological development in the past decade has brought us to this situation. One big step was the development of cheap hardware and huge server farms. Rackmounted clusters in a single location can now handle big problems by parallel computing. Recent algorithms have revolutionised parallel computing by chopping problems into small bits, Google’s MapReduce being one good example. The third revolution is in mathematics itself. A combination of topics in statistics, probability theory and computer science has produced highly effective methods.
A few big IT companies hire top mathematicians in large numbers and let them work in their area of choice. Microsoft Research has turned this into a fine art. “You have to be a star to be hired to Microsoft Research,” says P Anandan, managing director, Microsoft Research India. “And we never tell them what to do.” But the company engineers have gained immensely by the presence of so many top-ranking mathematicians in their midst. “I am sure a lot of their ideas have gone into Bing, and it has made its search results at least as good as that of Google,” says Anandan.
Ravi Kannan, principal researcher at Microsoft Research, was a professor at Yale University and a winner of the Knuth Prize, the highest award in theoretical computer science. “Microsoft realised early on that research mathematics, the kind of mathematics you do years after a PhD, is very useful, although not directly to make a product,” says Kannan.
Abstract mathematics can be useful in unforeseen ways. The world, as we perceive it, is in three dimensions, but the mathematics of space in 10,000 dimensions is useful for big data analytics. Kannan does some of this work in his lab in Bangalore.
Also working on abstract mathematics is Madhu Sudan, formerly a professor at the Massachusetts Institute of Technology, but now at the Microsoft lab near Boston. Some years ago, he had won the Rolf Nevanlinna Prize for his work on probabilistically checkable proofs. “Mathematics gives deeper solutions to traditional problems,” he says. Among the problems he has tackled is the mathematics of communication, especially the challenges of modelling uncertainty in communication. Engineers, interested in sending bits accurately, are focused on reliability of communication. Madhu Sudan focuses on the meaning of the messages, and investigates whether the sender and the receiver understand this meaning.
Another set of mathematicians, known as complexity theorists, is looking for simple ways to understand complicated stuff. At the New England Complex Systems Institute in Boston, Yaneer Bar-Yam looks at social problems through the lens of mathematics. He has published a study with a startling conclusion: the Arab Spring may have been an illusion. This is because societies are sophisticated structures that evolve like organisms, and revolutions disrupt the complex web of dependencies and create simpler systems. “Constructing a theory of governmental change from the perspective of complex systems can explain and perhaps anticipate the outcomes of revolutions,” says Bar-Yam.
Mathematicians have finally crept into even one of the most non-mathematical of entities: societal change.
Figure It Out
Who is an applied mathematician?
Anyone who uses mathematics to solve problems. In practice, it is someone who has advanced degrees in mathematics or a related discipline, and combines mathematical techniques with their domain knowledge to solve industrial problems
How do you become one?
By getting a master’s or a doctorate degree in applied mathematics, computer science, statistics or physics
Who needs applied mathematicians?
Finance companies, internet companies, analytics companies are big recruiters. Also, software, life sciences, manufacturing, among other industries look to mathematicians
What is the salary like?
The median salary in developed countries is slightly below $100,000 a year. In India, start-ups employ applied mathematicians at salaries higher than that of software engineers
What does the future hold?
The demand is increasing. The Bureau of Labour Statistics, US, expects jobs for mathematicians to grow 22% till 2018
Big Data Crunchers
As we collect and store huge amounts of data, scientists are developing mathematical techniques that can analyse this data. Most of them are at the confluence of mathematics and three different fields — statistics, computer science and physics Mathematics in Action
Analysing Real-time Data
Infections in ICU is a major reason for infant mortality. IBM has developed a model to monitor 4,000 physiological parameters every second. It predicts an infection 24 hours before symptoms manifest
Thinking in Many Dimensions
Some Microsoft mathematicians study space in 10,000 dimensions. It turns out that 10,000-dimension space is useful to analyse network traffic
Simplifying it With Complex System
The New England Complex System Institute analysed the euro crisis and concluded that the crisis is real but market overreaction drove up the interest rates making politicians too to overreact
No comments:
Post a Comment