Monday, May 6, 2013

Graduates: Data Mining as an IT Career?

 You purchased a book from Amazon last week. This week, you receive emails from them recommending some other related books that people are also purchasing.  You are wondering how does Amazon know what related books to recommend? Well, they uses the process called affinity analysis for cross-selling to recommend to you other products based on your purchase and the history purchases of others, who bought similar products. 
Affinity analysis "is a data analysis and data mining technique that discovers co-concurrence relationships among activities performed by (or recorded about) specific individuals or groups." Nowadays, as the use of big data increase, where a large unstructured  data  are captured, retailers use this process to perform market basket analysis to understand their customers' purchase behaviors and turn the information into insights. They then use these insights for the purpose of cross-selling and up-selling to drive sale volume through sale promotions, store designs, and discount plans.
Data mining, what is it?
Data mining is the process of collecting, searching through, and analyzing a large amount of data in a database.  It is known as the process of analyzing data from different perspectives and summarizing it into useful information, which can then be used  to increase revenue, cut costs, or both. 
Marketers are faced with the daunting task of using data to precisely to identify their customers' purchasing patterns and product interests. Companies use data mining process to turn raw data that they collected into useful information, which provides guidance for them to strategize their marketing strategies such as increase customer loyalty, unlock hidden profitability, reduce client churns, and increase revenue.

Data mining, why is it important?
Data mining is primarily used by companies with a strong consumers focus such as retail, financial, communication and marketing organizations. It enables companies to determine the relationship among "internal" factors such as price, product, positioning, or staff skills, and "external" factors such as economic indicators, competition, and customer demographics. Also, it allows businesses to determine the impact on sales, customer satisfaction, and corporate profits. With data mining, a retailer could use point-of-sale records of customer purchases to send targeted promotions based on an individual's purchase history. By mining demographic data from comments or warranty cards, the retailer could develop products and promotions to appeal to specific customer segments.
For an example, WalMart is pioneering in massive data mining to transform its supplier relationships. They captures point-of-sale transactions from over 2,900 stores in 6 countries and continuously transmits this data to its massive 7.5 terabyte Teradata data warehouse. WalMart allows more than 3,500 suppliers, to access data on their products and perform data analyses. These suppliers use this data to identify customer buying patterns at the store display level. They use this information to manage local store inventory and identify new merchandising opportunities. In 1995, WalMart computers processed over 1 million complex data queries. 
Data mining: How can companies gain competitive edge?
Businesses are collecting their customers data everyday, through point-of-sale transactions, credit card transactions, web browsings, or sale promotions, etc. Far too many of them are sitting on loads of good customer data and do nothing with it.  With the use of advance technologies and programs, companies can turn these data into a gold mine of insights that they can use to gain a competitive edge over their competitors.  Here are a few benefits that data mining can offer:

1. Basket Analysis: this is also known as affinity analysis which allows retailers to quickly and easily look at the size, contents, and value of their customer's market basket to understand the patterns in how products are purchased together. Leading retailers are leveraging this process to develop more profitable advertising and promotions, attract more customers, increase the value of the market basket, and much more to increase sale volume.

2.  Sale forecasting:  By understanding the customers historical purchasing behaviors, retailers can look at the customer in their market and try to predict how many will buy in the future.  This help the management to determine a strategy and come up with ideas of complementary products to sell. For example, if I want to open a coffee shop, I want to know how many people, household, businesses are within the one mile radius of my business and how many competitors are within that range. This help me predict who would be my main source of customers that are likely to purchase from my store. sbcHVjyWASXXxzJnH9U
3. Database marketing:  By examining customer purchasing patterns and looking at the demographics and psychographics of customers to build profiles, retailers can create the products that will sell themselves.  Businesses can collect information by sending out surveys, subscriptions and questionnaires, and then target customers based upon this intelligence.
4. Market Segmentation: The set of collected data can be breakdown into meaningful segments of age, income, occupation, or gender.This segmented pieces of data can effectively in helping businesses in running email marketing campaigns, or SEO strategies.  This insight also help you understand your competition in those market segments.  This feature allows retailers to tailor their products and promotions that satisfy the needs of their target audiences.
Data mining for an IT career?

Data mining services are used extensively by businesses who have strong focus on their consumers. It has been a big impact today that these advance techniques in business intelligence have been helping many businesses success. It helps to extract useful information from plenty of data, which can be used for practical business analysis. Data mining is being used increasingly in business applications for understanding as well as predicting valuable information, like profiles of customers, industry analysis, etc.

Thus data mining is a lucrative career choice that promises continuous monetary rewards as you climb up the corporate ladder. It involves the creation of a mathematical formula based on the given data. A career in data mining requires candidates to analyze and summarize large amount of data and uncover links in customer behavior, identify significant information regarding product testing and even expose fraudulent behavior. A data miner focused on marketing needs assists upper management in making decisions about price and product placement by drilling into customer transactions and purchase history

Careers combining data mining and marketing are often advertised in job description as "business analyst," "data analyst," "data information analyst," and "marketing specialist." In order to be a successful analyst, especially if you are working on a data mining projects, having skill in statistics, information technology, data management, and data integrity issues is preferred.  Also, familiarizing with the use of data mining software will also give you a higher score in the employer selection process.

According to and, an average annual salary for a Senior Data Mining Analyst is $74379 base on statistics in the U.S. as of 2013. The highest salary recorded was $104,601.  The lowest salary reported was $45,615.  These figures will vary on a state to state basis as these are average across all 50 states.


Dealing with Big Data is in the future that most companies will have to encompass to stay on competitive edge of their industry, therefore, the need of professionals with analytics skills to help them extracting and analyzing the data is going to increase.  Data Analyst professional work in a variety of industries and focus on the analysis, importation, transformation or development of data.  More simply, they take any information they have regarding their organization and present it in a way that makes sense to anyone who needs to use it.

A Data Analyst's skills go beyond implementing ways to find and present data.  Professionals must be able to review and analyze large amounts of data.  They must also know how to look for the information and know how to use the database that maintain information.

Data Analysts have to be extremely organized people who are also proficient at using many computer technologies.  They have to master the use of analytical and presentation software in order to take information and present it to a user friendly manner.  In addition, having an excellent communication skills, interpersonal skills, and a degree in technology or statistics is also needed.

So, do you think a career path to become a Data Miner and Analyst is the one for you?