Probability serves as the mathematical foundation of Machine Learning, providing a framework to make informed decisions in uncertain environments.

Applications in Machine Learning

• Classification: Bayesian methods use prior knowledge and likelihood to classify data into target classes.

• Regression: Probabilistic models predict distributions over possible outcomes.

• Clustering: Gaussian Mixture Models (GMMs) assign data points to clusters based on their probability of belonging to each.

• Modeling Uncertainty: Techniques like Monte Carlo simulations use probability to quantify uncertainty in predictions.

Key Probability Concepts in ML

• Bayesian Inference: Updates the likelihood of a hypothesis based on evidence.

• Expected Values: Measures the central tendency of a distribution.

• Variance: Quantifies the spread of a distribution.

• Covariance: Describes the relationship between two variables.

• Independence: Variables are independent if knowing the value of one does not affect the probability of the others.

Code Example: Computing Probability Distributions

Here is the Python code:

import numpy as np
import matplotlib.pyplot as plt

# Define input data
data = np.array([1, 1, 1, 3, 3, 6, 6, 9, 9, 9])

# Create a probability mass function (PMF) using numpy and the data
def compute_pmf(data):
    unique, counts = np.unique(data, return_counts=True)
    pmf = counts / data.size
    return unique, pmf

# Plot the PMF
def plot_pmf(unique, pmf):
    plt.bar(unique, pmf)
    plt.title('Probability Mass Function')
    plt.xlabel('Unique Values')
    plt.ylabel('Probability')
    plt.show()

unique_values, pmf_values = compute_pmf(data)
plot_pmf(unique_values, pmf_values)