Unsupervised Learning involves modeling data with an unknown output and is distinguished from supervised learning by its lack of labeled training data.

Key Distinctions

Data Requirement

• Supervised: Requires labeled data for training, where inputs are mapped to specified outputs.
• Unsupervised: Lacks labeled data; the model identifies patterns, associations, or structures in the input data.

Tasks

• Supervised: Primarily used for predictions or for guiding inferences based on predefined associations.
• Unsupervised: Selects data associations or structures as primary objectives, often for exploratory data analysis.

Modeling Approach

• Supervised: Attempts to learn a mapping function that can predict the output, given the input.
• Unsupervised: Aims to describe the underlying structure or patterns of the input data, which can then be used for various analysis and decision-making tasks.

Common Techniques

• Supervised: Utilizes techniques like regression or classification.
• Unsupervised: Employs methods such as clustering and dimensionality reduction.

Data Labeling

• Supervised: Each data point is meticulously labeled with its corresponding output category.
• Unsupervised: Systems are left to identify structures or patterns on their own, without predefined labels.Formally, in an unsupervised task, we have a dataset $X$ from an unknown joint probability distribution $P(X,Y)$, and our objective is to understand the underlying structure of the data with only $X$ available. Conversely, in a supervised task, we have both $X$ and $Y$ available from the same probability distribution, and we want to train a model $f^*$ that minimizes the expected loss on unseen data, i.e., $\min_{f\in \mathcal{F}} \mathbb{E}_{(X,Y)}\left[ L(Y,f(X)) \right]$. Ultimately, the primary difference between the two is the nature of the available data and the corresponding learning objectives.