💡⏳ Mining deep into Data Mining - PART I ⏳💡

 "Necessity is the mother of invention"

The need for knowledge is the root of data collection, discovery, and analysis. To be precise, we could say that the current technological world is drowning in data but starving for knowledge. Thus, data mining comes in handy

What is Data Mining?

It is the extraction of interesting, non-trivial, previously unknown, potentially useful, patterns or knowledge from the huge amount of data.

Want to know the alternative names of Data Mining?

👉 Knowledge Discovery and Databases (KDD)
👉 Data or Pattern analysis
👉 Data archeology
👉 Data dredging
👉 Information harvesting
👉 Business Intelligence

Data mining is indeed a confluence of multiple disciplines mainly

👉 Statistics
👉 Algorithms
👉 Data visualization
👉 Machine learning
👉 Pattern recognition
👉 Database Technology

Why not follow traditional data analysis?

👉 Traditional analysis of data will not be able to handle tera-bytes of data
👉 High dimensional data add complexity to the analysis.
👉 Complex data such as social network data, temporal and spatial data, multimedia such as audio, video are hard to be processed in a traditional manner

On what kinds of data can Data Mining be done?

👉 Relational database and data warehouse
👉 Advanced applications such as Heterogenous databases, social network data, graphs, multimedia, WWW, sensor data, data streams, multi-linked data, spatial, temporal or spatio-temporal data

Functionalities of Data Mining:

👉 Classification and prediction

                 ➡ Construct models that can distinguish classes - classify countries based on climate, classify cars based on performance.
                 ➡ Predict any numerical or missing value based on data inputs - Predict the house price after 5 years, cost estimate after a certain period of time

👉 Cluster analysis
                
                  ➡ Data is grouped together to form a group or cluster.
                  ➡ Class label (ie) cluster name is unknown.
                  ➡ Maximize intra-class similarity and minimize inter-class similarity.

👉 Outlier analysis
                
                  ➡ what are outliers?  : Data points that do not match with the general behaviour of the common data. They generally stand out from other data points.
                  ➡ Can be noise or exception.

👉 Trend and evolution analysis

                   ➡ Regression-based analysis
                   ➡ Pattern mining
                   ➡ Periodicity analysis
                   ➡ Similarity-based analysis

👉 Statistical analysis, association, etc

Basic steps involved in KDD process / Data Mining:

👉 Selecting suitable target data from the pool of common data
👉 Pre-process data
👉 Transform data into a flexible format
👉 Mining the data
👉 Interpret or evaluate the data and analyse patterns
👉 Finally, obtain Knowledge 
                

Issues in Data Mining

👉 Protection of data and privacy issues 
            
                ➡ Sensitive credit card information is been fed into the prediction model for fraud detection.
                ➡ Domain-specific data mining is highly ineffective.

👉 Handling noise and incomplete data

👉 Background knowledge of the data is required

👉 Knowledge fusion - Integration of the discovered knowledge with the existing one may be difficult.

👉 Diversity of data types such as web, multimedia requires different mining techniques such as parallel, distributed and incremental methods.



                         💡 Let's keep mining deeper  !!! 💡




Comments

Post a Comment

Articles by Hemapriya

Comprehending the state-of-art Digit Recognizer dataset using machine learning

Image
   Handwriting Recognition Handwritten text recognition has been a challenge since the first automatic machines were required to identify individual characters in handwritten texts.Consider the five-digit ZIP codes on letters at the post office and the automation used to identify them.To sort mail automatically and efficiently, perfect understanding of these codes is required. Included among the other applications that may come to mind is OCR (Optical Character Recognition) software. OCR software must read handwritten text, or pages of printed books, for general electronic documents in which each character is well defined. But the problem of handwriting recognition goes farther back in time, more precisely to the early 20th Century (1920s), when Emanuel Goldberg (1881–1970) began his studies regarding this issue and suggested that a statistical approach would be an optimal choice.  To address this issue in Python, the scikit-learn library provides a good example to better understand th
Read more

Performance Analysis of Weather Data using Machine Learning

Image
   Weather and Climate Observations Weather data is primarily important for determining the climate of a region. Climate is determined by a number of factors.The formation and advancement of storm systems, the amount of precipitation an area gets, and the number of cloudy days are all influenced by air pressure, temperature, and humidity at various altitudes.These influences affect the environment on a local, international, and global scale over time. Why performance analysis of weather data is important ? The value of weather data analytics in human life is immense. Accurate weather forecasting is beneficial to the agricultural industry, tourism, and preparing for natural disasters such as floods and droughts. Weather forecasting has a lot of economic appeal in news organisations, government agencies, and industrial agriculture. Performance analysis of meteorological data: we can use weather and climate datasets to better understand and forecast the effect on shipping and logistics pr
Read more

Market Basket Analysis using Association Rule-Mining in R language

Image
Association mining is usually done on transactions data from a retail market or from an online e-commerce store. Since most transactions data is large, the apriori algorithm makes it easier to find these patterns or rules quickly. Association Rules are widely used to analyze retail basket or transaction data, and are intended to identify strong rules discovered in transaction data using measures of interestingness, based on the concept of strong rules.  Apriori uses a "bottom up" approach, where frequent subsets are extended one item at a time (a step known as candidate generation), and groups of candidates are tested against the data. The algorithm terminates when no further successful extensions are found. DATASET:  Groceries_dataset Let's code and analyse the algorithm 💪 👉 Import the groceries dataset 👉Explore the data 👉 Perform data preparation such as checking the Null values, normalising the format of data to numeric values and group the data of similar values
Read more