Dofficially D!

Finally, I put to official use the name that was given to me by the people I loved working with (read, Inductis colleagues) - D

The Diamond Consulting Case Competition on campuses is called DConstruct. I think they think its D for Diamond. (wink wink). Its D for D!! :)

Couple of links I could find about DConstruct - Here and here

Customer Analytics is here to stay

Couple of very interesting articles here - 1 and 2

The first talks about the relative unpreparedness of Asian Banks in maximizing the value of their data as well as the need for advanced analytics for the next stage of growth. Not sure if the link would work; So, I am pasting the article below anyway.

The second talks about digiton's view of an IBM report that talks about the end of advertising unless they start investing in customer analytics.

Two separate reports talking about the same thing - Customer Analytics is here to stay, and gradually its become the backbone of all businesses. Encouraging, isn't it?

****************************************

Copyright 2007 Business Wire, Inc. - Business Wire

November 9, 2007 Friday 2:00 PM GMT

HEADLINE: Survey: Asian Banks Seek Right Strategies and Decision Tools to Fuel Smart Growth in Today's Credit Boom;

New Survey from The Asian Banker and Fair Isaac Spotlights Need for Better Data and Analytics for Managing Risk, Fraud and Customer Acquisition

As credit markets in the Asia-Pacific region continue to expand at record rates, results of a new region-wide survey underscore a growing need for proven strategies and solutions that enable banks to make profitable customer decisions that accelerate growth while limiting risk and losses.

Commissioned by Fair Isaac Corporation (NYSE:FIC) and conducted by leading financial services research and intelligence firm The Asian Banker, "The Asian Banker Consumer Credit Practice Survey 2007" spotlights current practices in credit customer management, risk management and fraud control. The survey engaged senior banking executives, experts and practitioners in 10 major financial centers across Asia Pacific, including Singapore, China, India, and Hong Kong. The Asian Banker and Fair Isaac revealed the results in Shanghai this week, during Fair Isaac's first InterACT customer conference in China.

The survey indicated that as Asian banks work to expand their consumer credit businesses, they are in need of advanced decision-making capabilities to help them smartly capitalize on their growth opportunities. The survey found that most banks in the region are still at the early stages of building and leveraging sound analytic approaches and technologies to improve the processes that drive their most important customer decisions. Key findings of the survey include:95 percent of respondents identified establishing a reliable credit scoring process as a key operational challenge. Less than 10 percent of respondents currently utilize advanced techniques for fraud detection and prevention on a regular basis. Less than one-quarter of Asian banks utilize advanced customer segmentation techniques in their marketing activities. Less than one-third of banks incorporate customer profiling or profitability analysis on a regular basis.

The survey was conducted at a time of burgeoning growth in Asian credit and lending markets. According to The Asian Banker, the current $3.9 trillion consumer credit market in Asia Pacific has an estimated potential of $8 trillion.

"By investigating how banks manage key functional processes and customer decisions across the consumer credit lifecycle, The Asian Banker sought to examine the preparedness of Asia-Pacific banks to achieve their growth potential," said Dr. Grace Liu, Senior Researcher at The Asian Banker. "Specifically, we looked at their current capabilities as well as the fundamental challenges they encounter. Diversity of credit markets, varying levels of process maturity and different go-to-market strategies were key themes that emerged, along with the common challenges banks face in acquiring and leveraging quality customer data."

"We are pleased to partner with The Asian Banker to provide a timely view of current practices, perspectives and requirements for success in the Asia-Pacific credit industry," said JY Pook, Managing Director for Fair Isaac Asia Pacific. "The fast-emerging markets in this region hold tremendous opportunity for banks and lenders that approach it with both the right growth strategies and the right combinations of data, predictive analytics and sophisticated decision technologies. Fair Isaac's mission is to help Asia-Pacific banks embrace and leverage the strategies and solutions they need to consistently and confidently make customer decisions that enable smart growth."

Survey Details

Select findings of the survey include:

Data, Credit Scoring and Fraud Analytics are Keys to Growth

The Asian Banker found that the ability to capture, analyze and act on customer credit data is a key determinant for success in the lending origination process. Respondents were concerned by the state of data, predictive modelling capabilities and account automation across the consumer credit lifecycle. They indicated that an inability to effectively leverage data is an obstacle to developing advanced decisioning capabilities, and thus is an obstacle to consumer credit business growth. Eighty-seven percent of respondents indicated that making effective use of credit bureaus is challenging, while 88 percent indicated that they have difficulties achieving effective automation of credit approval processes.

In the areas of fraud detection and prevention, the survey found that banks rely largely on basic fraud detection methodology, and view fraud detection and prevention improvements as secondary to growing customer acquisition and account management capabilities. While 89 percent of respondents actively use internal or industry-shared "blacklists" - the most basic of fraud detection methods, only 22 percent use more advanced modelling methodologies such as neural networks and profiling.

A Need for Deeper Understanding of Customers

The survey illustrated that Asia-Pacific consumer credit markets are at different stages of development, with varied levels of market penetration. Emerging economies like India, China, the Philippines, and Indonesia demonstrate the highest potential for market development and growth, while more mature economies like Singapore and Australia comprised the group with the highest ratio of penetration and indebtedness.

This diversity is further indicated by different approaches to market development. In emerging economies, banks such as those in India and the Philippines (or 29 percent of all banks surveyed) utilize broader-based approaches to customer acquisition, while banks such as those in Singapore and Australia (24 percent of banks surveyed) have adopted more advanced marketing strategies in customer segmentation. However, fewer than half of all banks surveyed have established mechanisms for measuring the effectiveness of marketing campaigns, due largely to challenges in quantifying and establishing appropriate performance measures.

China Poised for Growth, But Needs Advanced Analytics

According to The Asian Banker survey, China is one of the highest-potential growth markets. However, the country is still at the nascent stages of market and process sophistication. For example, almost 65 percent of Chinese banks surveyed agreed that they were unable to target appropriate customers for their consumer credit offerings. More than half of Chinese banks also viewed the effective use of credit bureaus and historical data a major challenge.

Evaluserve report predicts Indian KPO Boom

This link here [may require subscription.. not sure] talks about the impending boom in Indian KPO market. Headline says - India to dominate global KPO mkt; create 1.8 lakh new jobs

"The Evalueserve report also states in detail about the few sub-sectors within the KPO industry that are expected to do well. These include banking, finance, securities and insurance research, data mining and analytics and contract research organizations and biotech services."

Thats some good news ;). Having decided to make a career in this field, and still watching it find its firm feet, I think these occasional headlines are very importants for us to feel confident about the career choice we have made.

Nascence, Idling & Adolescence

One of the things I’ve always been scared about, in an organization, is the transition from Nascence to Adolescence. What I mean is a new/small/fresh/startup environment of the firm (NASCENCE), gradually maturing into the processes/culture/large company environment (ADOLESCENCE). However, the theme here is the transition process and my fear around Idling. Often, as we build delivery capacity, the lag between sales and delivery often leads to idle (delivery) time for the team.

Let me show a cycle of events, which I hope some of you will identify with –

1. Company gets a project in a new space (analytics here)
2. Company hires people to get it done
3. Company views this as an opportunity to build its presence in the space
4. Company starts building the business case. A core team is put together to convert the project into a vertical/business
5. The wheels start rolling. Sales team is roped in for selling. Delivery capacity starts getting built. Everyone is busy. Everyone is enjoying the dirt on the track.
6. New guys come in. Some start working on new projects. Some wait for projects.
7. The unutilized team members are put on firm-development and intellectual capital development
8. On the ground, there is an uncomfortable buzz. The seriousness required to complete these internal initiatives is often missing. The unrest begins!

I feel that the initial lot of people are usually overworked, because they came in after a project was sold, which, in high probability, led to the identification of opportunity. The IC and FD guys seem to be creating unrest. Why?

My guess is that the blame should be taken by the initial overworked guys like me, who end up believing that they are the ones doing the “real” work, all FD and IC is just a way of keeping guys busy.

The second group to take blame should be the leadership which is in charge of looking at the FD and IC initiatives. Its their responsibility to inculcate the sense of pride, responsibility and importance associated with these inward oriented projects.

The third group to be blamed, to the least extent, is the new group itself. A simple saying like Rome wasn’t built in a day goes a long a way in building organizational maturity. We all need to realize that everyday cannot be a perfect 9 hour work day. Just as there are bad days/weeks of 16 hours+ work per day, there are bad days/weeks of 0(ZERO) work per day. Just as you need the client work to generate revenue, you need FD and IC initiatives to build the backbone of a firm. The initial chaos of excitement needs to gradually mature into a process driven organization meant to meet the needs of a large number of people working together. The ad-hoc decision making needs to be replaced by structures that are ready to take the load of a large number of people demanding individual attention.

That said, I believe that it is how well we handle this stage of growth, that differentiates a great leadership from an average leadership. and yes, one of the ways of doing it is communication (clear, effective and copious)

What do you think?

Desktop Analytics

Read this quote - (Measuring productivity, or the lack thereof, on the personal computer. "Desktop Analytics") on Google Reader feed of Data Sciences Analytics

Interesting. Not getting into the contents of the real post, I loved the statement. Its funny how many times I have tried to measure my desktop productivity. My desktop (or, laptop) time includes work, watching cartoons/animations/manga, reading comic books, reading blogs, reading general stuff, mailing, searching for tid-bits, playing games, etc etc. Amongst all this, I have no idea if my desktop productivity is 50% or less than that! Oh Yes! I would like to believe that I am terribly busy and am doing a lot of work, but the fact is that, ignoring the dependencies of a workplace, my productivity in the last several years would never have been more than 50%. On a really bad day, it might be 100%. But that would have to be really bad day!

All ye data folks out there, is there a way to capture this data? Any tool? I am sure the results are going to depress the hell out of me. BTW, this blog post - would it go under productive use or unproductive use? My guess is that its unproductive use, unless people find a skewed logic of calling it a break to refresh my decaying brain cells and hence, increasing my productivity. And a break is a part of a productive work day, isn't it?

Communicating is required!

A very impressive experience based blogpost by Kevin. It rightly emphasizes the equal if not greater importance of communicating over crunching!
I guess its true not just in an internal vs. consultant kinda scenario, as Kevin mentions, but also in the case of a Manager-Analyst relationship. There are times when I have seen the Engagement Managers and above have a tough time with the team, because they want to hear a lot of things, but not just the numbers! ;)

My first white paper.. on rapid market segmentation.

My first ever White Paper is here titled Rapid Market Segmentation: Collaborate and Conquer

Download it from the link for a rapid reading. There are some related articles on Diamond's Analytics blog as well

Let me know what you think.

Back from vacation

There folks! Just came back from a 2 week break (spent partly in my village, and partly in Chicago).

Will be right back with more and comments!

Analytics Start-Up Series : 3 of Many

I have covered my first couple of posts on this topic here (1, 2)

Coming to the fourth pillar of Strategic Alignment – Organizational flexibility is the most important to have and most difficult to ascertain. Whether a business or a group of individuals are ready to do something that they have never done before, is a lot of introspections. No offence meant, but not every leader is Captain Kirk, nor every business USS Enterprise. At the same time, isn’t flexibility at the heart of entrepreneurialism?

Nevertheless, what did I see in my experience? High inertia to change (But we have been doing it this way, and it works!), traditional ways of thinking (been there, done that!), skepticism (will it work?) and arrogance (what can this new kid on the block tell me that I don’t already know about running a business?)

Inertia and traditionalism are vices that cannot be sorted out in the short term. I always felt the need to know what I am up against. A little better. Even with a small firm, I liked figuring out where the funding is coming from. If the money guy wants you to generate returns yesterday, then you are in a big soup. Very recently, I heard about this new start up (a few guys running off from a fairly known name) where the Money guys have given them a 3 year window on returns. I have my reservations about the exact terms & conditions. But nevertheless, that’s a flexibility that you need to pull it off.
One of the biggest vice and virtue of youth is the need to break the rules. And those who play by the existing rules are not the rainmakers. What used to really bug me off was a reminder to stick to the old rules to set up a new business. My usual retort (in my head) used to be – Ok! So you know how it works! Why don’t you just send us a 10-step guide to building a 1bn dollar business? That will help me save time on thinking! BECAUSE THERE IS ANYWAYS NO GODDAMN POINT OF THINKING! ;)
I learnt to think politely later on!  Waiting for crisis to drive a point home seemed to be the easiest way out! But then, business are not built on decisions taken during crisis!

Lesson: Look at who you’re going to work with, and under! Rule of thumb for start up – If you have too many guys who started off as the ground level guy and became Project Managers before becoming leaders, and have not taken a single weird decision in their life, are very unlikely to flexible. Almost every great leader CV has something zingoistic! Remedy for disaster – A “did my schooling from ABC (a known school” and “went to IIT and then IIM” and then worked for company A/B/C (all known brands) for x years (x>5). Whoops! You know that they have already lost their creativity!


Skepticism and Arrogance – Skepticism can be handled. Arrogance, cannot. Unless you are ready to fight fire with fire! Are you one of those people who, when pinned against the wall, can poke their nose into the poker’s eye before betting your life on why you think something is going to work! For skepticism, a thorough research is very important. That’s something that gets discounted when you are in a hurry. Young people like me always make the mistake of not having thought through all possible questions. We just find it too boring, and we ourselves are too arrogant. I know it. I can handle the questions. But vacuous answers increase the amount of skepticism.

Lessons learnt: Do your research well. And be ready to fight for it. Nothing comes easy. You have to be the bad guy at times!

Analytics Start-Up Series - 2 of Many

I will continue with part 2 of the start up series and focus on the third part of the element wheel - New Product Design (Air)


New Product Design

What? By the time I had joined the team, we were still defining what we want to do, and what we don’t want to do. Analytics has come to connote not just data mining and predictive analytics, but even research analytics, product based analytics, dashboards, etc. We wanted to focus more on the marketing analytics, and that’s why we were MCoE.
However, the problem/uniqueness of our positioning was that we were focusing on a process driven analytics approach, while 90% of the listeners with different levels of analytics’ understanding had thought largely of product driven approach only. Even “SAS-skills” were “SAS” skills.

For whom? Moreover, we needed to decide which all verticals we will build our presence in. That’s tricky. Being small and new, you want to prove a point. You are ready for any project that comes your way. However, if you do your first project in healthcare with your key focus being FS, the next time you go to a client, you have a healthcare case study to talk about. If it’s an FS client, you neither want to own the case study, nor disown it. Boom!

Why? The bottomline for a sales guy remains – why would someone buy what you’re selling? Is there an identified need? Is it expressed? Would you need to educate the buyer? In the Indian market, for instance, Fractal Analytics, I think, has done a great job of educating the financial services sector about the need of and opportunities for analytics. There are similar examples elsewhere and in other industries too. Having said that, if we look at the analytics market today, the education is taken care of. The market does have an expressed need for analytics. If we take a closer look, the first industries to adopt analytics were financial services and telecom. And both these sectors loved keeping their data to themselves. They built strong in-house teams. However, things have changed in the last few years as firms have started engaging third party vendors (just as they adopted consulting/consultants as third party unbiased experts with a broader view) for analytics. But today, a lot of other sectors including Public Sector, Healthcare, etc. have emerged as buyers of analytics. Marketics, for instance, had more depth in FMCG than FS, given the ex-P&G background of its leadership team

Where? From a third party vendor point of view, almost everyone wants a piece of the US analytics market. The other markets have been slower to adopt analytics outsourcing. The other truth is the relatively crowded analytics vendor market in US. Net result, apart from some of the early movers like Fair Isaac, not a lot of vendors have been able to build a large scale. However, my MCoE stint taught me that there is a significant opportunity lying the Asian and European market as well, if you have the right connections, credibility and content.

Lessons learnt – Identify a market that you understand well, and where you have the credibility to sell. Sell only a bit, and understand it in depth, and avoid trying to be everything to everyone.

Analytics Start-Up Series - 1 of Many

Yesterday, I was talking to one of my friends who wanted to get my perspective on what did I learn or not learn about starting an analytics company (having been a part of multiple start-up environments). That’s when I thought tt might be a good idea to pen down my thoughts on this.
Disclaimer: This applies to my understanding of a company which is doing offshore analytics.

I always looked at my learning along four dimensions


This post is going to be the first in a series of posts where I will write about my experiences.

1. Strategic Alignment With Overall Business

I am drawing largely from my first company experience here, which was with a very large Indian IT firm thinking of setting up an analytics practice. When I joined the team, Gayatri Balaji was leading the initiative, and she had Dhiraj Narang helping her. Me and Shivani Sohal joined her as part of our third training stint. Apart from Gayatri, the three of us were raw with no analytics background/experience.

Analytics, or Marketing Center of Excellence (MCoE) was a prized initiative of the company at that point. It was an attempt to move up the value chain by doing “intelligent” work (in the Business Intelligence way, not that the company was not doing any intelligent work!).
However, that being said, what our small and inexperienced team (with the exception of Gayatri) soon realized is that its one thing to say that we want to do “this”, and another to align it to the overall business.

There are four set of challenges that we discovered –

A. Existing product portfolio

Context : The company already had a BI practice, a CRM practice, and a lot of analytics was already being done in different relationship pockets. To put it bluntly, every time a relationship needed someone with “SAS skills”, they hired one and put him/her on the relationship. No need to aggregate the “SAS skills”, (which is what analytics job postings have reduced the required analytics skill-sets to!). Additionally, tools like SAP and Oracle have their own analytically intelligent layers, and SAP and Oracle are separate practices within the organization. Imagine your plight when you’re talking to a client about analytics and she says – “Well! But that’s pretty similar to what you’re BI team talked about. They probably had a higher product focus, though!” And you start looking at the account manager, who has probably introduced every practice to the client (to grow the account). It was not our fault that we were the new baby on the block. Interestingly, the leadership had never thought of aggregating the knowledge lying here and there in the firm to have a solid ground from the beginning.

Lesson Learnt: If you’re going to cannibalize your existing product line, you need to be sure on what you are offering, why you are offering, and how will you work with the existing product lines.

B. Stakeholder Alignment

Context: In the organizational power play – strategic positioning can mean that you are the weakest (fresh out of the closet) player, or the strongest player (the whole company is looking at you). Usually, you are not stuck in the middle. If you are the strongest, the performance becomes extremely short term oriented. Stakeholders want to see quick wins, proof of concepts and a latent potential (as visible through a practice bursting at the seams!)

a) “Who’s with you?” - We realized pretty soon that very few important players have been sold the concept and its importance to the overall soon. The attitude towards the new practice ranged from “This is neat!” to “Oh! So we are wasting money on analytics this time”. To an extent, the varying levels of cynicism is expected in large organizations. The problem we faced where cynicism in the mind of decision makers/policy makers.
b) “Who gets the credit?” - Driving from my other experiences, analytics team can potentially be at direct conflict/synergy with another practice. For example, an offshore analytics center (analytics outsourcing) model can be a potential threat as well as support to analytics consulting. Analytics (platform independent) can be a threat to product driven analytics, but can be used to augment the nature of analytics as well.
c) “Who gets the money?” – Given that analytics is a horizontal solution and not specific to industry, revenue recognition is always a challenge. All the verticals stake claim to the analytics revenue, while analytics unit may have a separate revenue target. For instance, a 100MM target for Financial Services vertical will be achieved through products, services, analytics, implementation, etc. However, the 20MM analytics target will be achieved through a combination of work done across verticals- such as Financial Services, healthcare. Every dollar generated by analytics team will be claimed by the respective vertical. However, the effort devoted to sell analytics will be lower, because there is no specific FS-Analytics revenue target.

Lesson Learnt: If you’re an outsider roped in to run a new business initiative, make sure that you understand the powerplay and relative buy-in. Understand the weak and strong points of everyone you’re compulsorily going to deal with. Equally important is to understand the relative aspirations that help you share, transfer credit of the work done in a politically correct manner. Sales is a tricky issue that we will touch separately later

(...to be continued)

A weekend thought on analytics

I have often thought about what is the kind of analytics I want to do and tried to correlate it with the kind of analytics that companies are thinking of doing.

• I want to think of analytics as a key to strategy. Organizations want strategy to define the analytics required for the business.
• I want to do analytics to optimize social benefits. After all, what is policy but a goal programming issue! Policy makers are driven by soft objectives and non-economic intentions more than economic incentivization. They ignore data for convenience.
• Analytics can, potentially, be used for wealth allocation. Just as Activity Based Costing helps organizations decided the key contributors and key unproductive consumers of resources, good analytics in the social context can help us define how the resources should be reallocated.
• I want to do analytics on product development. I can see that organizations validate product ideas using analytics more often than analytics for coming up with ideas. Aren't there creative people who can think of random surveys already done by equally random agencies and come up with new product ideas?

In short, my ideas on analytics have a pretty strong (negative) correlation with the reality!

More on Sports

Its surprising that the sports and analytics phenomenon should get so much press coverage suddenly.

I saw an article in viewpoint , the journal of Marsh & McLennan which again talks about the Maths of sports.

Some of my previous posts on this topic are here - 1, 2 and 3

Analytics and Invasion of Privacy

Lately, I have seen a string of articles on the concerns around Department of Homeland Security's A.D.V.I.S.E program (Analysis, Dissemination, Visualization, Insight and Semantic Enhancement). People have had concerns around invasion of privacy and DHS acting like a peeping tom.

A similar set of concerns came around the DoD and Microsoft deal for analyzing Electronic Patient Records.

“Dr. Deborah Peel, chairwoman of the Patient Privacy Rights Foundation, views the patient information not as a goldmine ripe for exploitation but as a collection of personal and sensitive health information that needs to be zealously guarded and only accessed with express consent by the patient.”

This blog here raises an interesting point

“data mining by definition compromises the privacy of people represented in that database - if your personal information is included, there is no way to opt out.”

The least that can be said is that the concerns are valid. But having said that, here is what I think of the problem of privacy invasion -

1. What information is private information? When a customer signs up for something like a loyalty card and agrees to give information about their demographic profile, income, tastes & preferences, this is voluntary sharing of information. What they also realize, in addition, is that their purchase behavior can be tracked on the card (that’s how they earn loyalty points which are redeemable against tangible benefits). This kind of information, according to me, is not private information in the strictest sense (for the organization which has collected this data painstakingly).

3. What could be called privacy intrusion? Even though google has tried to bring some changes, what it usually does (I am talking about simple examples like ads next to your emails when you are accessing gmail accounts) can be called fairly intrusive.

2. Can private information be kept private? If we don’t associate a piece of information with a named individual in a database, keeping the name or the identifier as a random number, the analysis insights at the end of the day tell me a profile. Individuals with attributes a, b and c are more likely to behave in a certain manner. At this stage, we still don’t know who has attributes a b or c. This step is critical to upholding user privacy.

4. What can organizations do? As a third party analytics services provider, we must realize that data security standards need to be absolutely non-negotiable. This requires

a) working only with masked data,

b) removing information that helps identify an individual to as much extent as possible,

c) maintaining high security standards while transferring/porting data

d) create an onsite-offshore delivery model where data security concerns are alleviated by working onsite for some time and creating master data tables that alleviate data security concerns.

5. How big is the problem? Well, as any analytics provider will tell you, the real value of information is not in “who”, it lies in “what”, “how” and “why”! Once an organization has answered these three critical questions, “who” is the final step of the strategic gameplan, and can be answered at a group level, rather than individual level.

Having said this, projects like ADVISE are bound to create a fair bit of skepticism around the way private information will be treated, and the impact of lying in the group of false-positives (being identified as a terrorist when you are not one!)

Analytically Sports... Continued!

FractalAnalytics seems to be a step ahead of me! Here is a news item that covered their prediction on the First Match of World Cup (Cricket), between West Indies and Pakistan. Lo and behold, 2 of the predicted scores match exactly! Nobody would have expected that granular a performance prediction to be correct 75% of the times (as per their claims).

Going back to some of my earlier posts on use of analytics in sports - at Diamond Analytics Blog and here itself, what Fractal has already managed to do is a proof of concept.

The factors that I don't see them looking at is the location/playground/weather/batting order/ bowling order/ etc., which do have a big impact on performances.

> Under overcast conditions, the chances of Indian batsmen holing out to the wicketkeeper goes up significantly.
>> The chances of genuine swing bowlers running through the side on grassy pitches is high
>> On flat tracks, against minnows, in subcontinent kinda pitches, batsman have a feast day

These are examples of hypotheses that can be tested using data.

It would be interesting to see how teams can use a model like this to decide team composition, play batting orders, etc.!

Complex Data to Complex Knowledge

Dell Zhang quoted the challenging problems in Data Mining research [ICDM ‘05’].

It will be interesting to touch upon each of these problems in greater detail. However, for now, the most interesting bit is 4. Mining Complex Knowledge from Complex Data. That is what defines the heart of data mining for me.

Mining – From its origins in extraction of minerals, mining has traditional implied extraction of extremely valuable stuff from earth. Wikipedia says any material that cannot be grown from agricultural processes, or created artificially in a laboratory or factory, is usually mined. What is implicit here is the application of intelligence for achieving this feat.

What organizations are increasingly finding difficult to do is to revisit the (apparently) already mined data and come up with new strategies. And when we say already mined, most organizations find it difficult to let go of the semi-cooked analysis that might have been done to meet immediate requirements of marketing executives breathing down the neck of analytics departments.

Complex - A complex is a whole that comprehends a number of parts, especially one with interconnected or mutually related parts. [Wikipedia]

For most of the organizations today, integrating parts of information to see the bigger picture is the new challenge. Today, strategies are not being formed at department level and there is a higher need for departments to come together for an integrated strategy. A perfect example would be the need for IT, Marketing, Customer Services and Products team to work together for an end-to-end customer offering.

Data to Knowledge is the heart of analytics and there can be a host of tools used for traversing the distance.

Like every problem solving exercise, Data Mining and Analytics is an extremely structured exercise involving a series of rigorous steps

1. Business Understanding – involving setting the context and defining the problem to be solved.
2. Data Understanding – which involves getting a sense of the data that is available, that can be made available, and that needs to be available for solving the problem
3. Data Preparation – One of the most important and rigorous steps of an analytics project, this involves bringing various data elements together and creating a data story. Understanding linkages between various data sources, their integrations and disintegrations, tying them with the problem objective to create new variables, vintage of data, changing shape and design of data capture at the enterprise level are all seemingly tedious but life-saving checkpoints!
4. Modeling/Segmentation/Solutioning - This is the point where the wheat is separated from chaff. Having got your data together, can you use the appropriate statistical and analytical techniques such as cluster analysis, regression, neural networks, et al. to solve the problem at hand. The solutions here range from simple reporting dashboard to complex algorithms that are not easy to explain.
5. Validation & Deployment – A true romantic movie is never over unless all the things have fallen into place. We need to be able to establish beyond doubt that the results are accurate. Predictive modeling projects have been known to use advanced validation techniques such as coefficient blasting, in and out of time validation, sensitivity analysis, bootstrapping, etc. Deployment faces a different set of challenges in being able to replicate the solution on a production server for ongoing maintenance and reporting.
6. The key stakeholder buy-in – This is a step that everyone overlooks as part of the analytics lifecycle. However, this step has nothing much to do with analytics apart from making sure that the first 5 steps are correct to the last dot and cross and is well documented for everyone’s reference.

That’s where the sermon of Rabbi Amit gets over.

Acquisitions - Offloading Offshore Analytics

This one merits a mention

- WNS is acquiring Marketics, an offshore analytics firm founded by some ex-P&G CMK senior professionals. People like Shankar Maruwada, Ramki, etc. are extremely smart people. A $65 MM cash deal for its 200 people, with $30MM upfront and $35MM earnout in an year, does show the smartness of the deal.

Isn't this somewhat an action replay of how Inductis was taken over by EXL (another large BPO company in India). Did things really change at Inductis? Not a lot. There were a lot of positive reinforcements, with a few bad things. The worst being the uncertainty around people's career.
The best being a sudden n-fold increase in sales staff. The to-be-debated item of the roster was "can BPO sales guys sell analytics projects, which are so specialized skillset driven ?"

My friends at Marketics (like The Other Side) - I wonder what they are feeling now.

That aside, why do I think this is an important news item -

1. The role of offshore analytics has gone up tremendously - Examples - Inductis doing it in 2001, Companies such as Marketics, Modelytics, MarketRx, Absolute data, Zeus Associates, etc. have been catering to analytics needs of companies across the globe.

2. A lot of these companies have done tremendously well as start ups and have managed to build the first level of growth. However, to hit the next level, most of the companies will need a higher amount of funding. That comes either through PE firms, VC firms, Investors who may want to fool around with the way the company is being run. Or, through buyouts like the Inductis one, where the provider continues to work as an independent company. But the acquisition is more often than not driven by the need to have more funds.

3. Most of these companies were set up by young, ambitious individuals who spotted an opportunity in their respective consulting/core organizations (Pharma folks starting Market Rx, MMG consultants starting Inductis, P&G folks starting Marketics). A lot of these people probably are looking for big money (one of the drivers of entrepreneurial fire is money). Its simpy a question of timing your investment. Well, sell high!

4. Does the scale of the game change for the offshore analytics providers????

Well.. lets keep talking about these!

Whats with going solo?

Avinash Kaushik writes stuff about web-analytics that can be used by novices as primers for building their understanding, and experts as content matter for strong debates. Avinash announced on his blog that he is going solo and will be a analytics evangelist for google for his first assignment.
Kevin Hillstrom of MineThatData has also decided to go solo. No need to tell the readers who Kevin is and the analytical depth with which he writes.

Now, lets Mine This Data, pun unintended! :)
A. The analytics market is growing insanely. There is need and space for a large number of such strong SMEs as Kevin and Avinash.

B. A thorough understanding of analytics is a fairly complex skillset, and rare. Anyone can come and talk data and profiling and dashboarding and modeling. But there aren't too many people who understand the complete data analytics process well. Right from the vision, method, depth and technology for data acquisition to the expansive business application of analytical frameworks, while maintaing a sync the IT and Business startegy of the firm, is no mean task.

C. Most importantly - Intellectual bankruptcy. The number of analytics blogs that have come up, with people writing on specific subjects (Avinash on web analytics) to people writing on all analytics subjects (Kevin), is proof enough for me that the bubbling energy amongst all these intellectuals needs a vent. While blogging helps them think more, and beyond the scope of their day to day assignments, going solo implies they are ready to get dirty once again.

What do we have then - Market <> Skillset <> Desire! What comes out of it at the end - Pure Magic. All the best Avinash, Kevin!

Avinash - if the quality of your posts go down, I will start throwing hate mails at you! :)

Houston Rockets using Analytics

Recently, I had posted my views regarding the use of analytics in sport (on the company blog here).

I just came across this additional piece of information today. Rockets using Daryl Morey and his analytical skills for the game.

It justifies my faith in the world of analytics and its universal applicability.

UNC and SAS partner for advanced analytics degree

University of North Carolina and SAS are partnering for an advanced degree in analytics. This is pretty cool stuff, I say.

So far, most of the people I know in the industry have started off with theoretical knowledge of statistics and econometrics and then built their understanding of "analytics" which is beyond the subject matter. I am hoping that courses like these would help fill the void

Crime Analysis Blog

Here... I want to link it to my RFID post .. Crime Analysis using RFIDs!!

Thanks Sandro for pointing me to this...

Hilarious (Though not quite analytics)

Just saw this post on Juice Analytics.. here

Hilarious.. Outrightly hilarious and contemporary. We don't even realize how close we have gotten to the evil/angel called PPT presentation for all our communication needs. And the breakup surely is gonna hurt, as and when it happens!

Tagging Frequently

One of the most interesting possibilities with analytics is what I discovered in my first job, working with the RFID team. While we were discussing RFID tags, the way they store data, and the nature of the data stored, we were also discussing the umpteen CVM and Loyalty possibilities opening up in retail. The next idea that came about was RFIDs and Cars and tagging for payments, tolls etc. And before we could even blink our eye, we were talking about theft prevention using RFID. I am sure its a topic beaten to death by now (we were having this discusion in the september of 2003!)

However, RFID and theft prevention is basic. Its easy to track vehicles based on their tag. But based on RFID tags and the data stored on that, lets see what the possibilities are -

1. Theft Forecast - based on users/ forecast zones/ theft trends/ etc.
2. Traffic Forecast - optimize the flow of traffic to key destinations by dynamically managing the routes.
3. Car designs by user segments...

and I am just warming up! :)

Let me introduce myself

Well, I am Amit Das, a Consultant with the Analytics team at Diamond Management & Technology Consultants (erstwhile Diamond Cluster International/ Diamond Technology Partners) (NYSE: DTPI). The company also has an analytics blog carrying some neat discussions!

Given my numerous dealings with data, analytics and data analytics, and the gazillions of terabytes of thoughts that keep coming to my mind, I just decided to create an alternate blog!

My other frivolous existence can be found here