TensorFlow, an open-source framework developed by Google Brain, has become a leading tool for machine learning and other computational tasks.

Founding and Evolution

TensorFlow stemmed from Google’s internal proprietary tool, DistBelief. In 2015, the firm made TensorFlow available to the public, empowering researchers and developers with a wide array of capabilities through an intuitive and structure-driven platform.

Key Components

• TensorFlow Core: The foundational library for building machine learning models.
• TensorFlow Layers (tf.layers): Offers a straightforward method for constructing and training neural networks.
• TensorFlow Estimator (tf.estimator): Streamlines model deployment through high-level abstractions.
• TensorFlow Keras: Facilitates quick and efficient model generation using high-level APIs.
• TensorFlow Feature Columns: Aids in defining input functions for model training.
• Explanability & Fairness Toolkit: Enables comprehensive model evaluation from the fairness and ethics perspectives.

Usage Scenarios

• Multiple Devices: Effectively executes tasks across CPUs, GPUs, or even distributed environments using tf.distribute.
• TensorBoard: Visualizes model graphs, loss curves, and other metrics for real-time tracking.
• TensorFlow Serving: Streamlines model deployment in production setups like servable, which separates the interface from the model itself.
• TensorFlow Lite: Tailors models for resource-constrained devices like mobiles or IoT gadgets.

Licenses

• The core TensorFlow is distributed under the Apache License, Version 2.0.
• Supplementary libraries and tools often come with their separate licenses.

Despite the expansive library of tools, TensorFlow’s modular structure allows for a choose-as-needed approach, making it popular for both academic and industrial applications.

TensorFlow offers a rich set of features optimized for highly efficient and scalable machine learning workflows.

Key Features

1. Graph-Based Computation

TensorFlow models computations as directed graphs, where nodes represent operations and edges represent data arrays (tensors). This architecture enables multithreading and distributed computing, allowing for parallel setup and execution.

2. Automatic Differentiation

The framework provides tools for automatic differentiation, which is crucial for computing gradients in various optimization algorithms, such as backpropagation in deep learning models. TensorFlow’s GradientTape is a popular mechanism for tracking the operations that manipulate tensors and computing their gradients.

Here is the Python code:

import tensorflow as tf

# Example with tf.GradientTape
x = tf.constant(3.0)
with tf.GradientTape() as tape:
    tape.watch(x)
    y = x ** 2
dy_dx = tape.gradient(y, x)
print(dy_dx)  # Output: tf.Tensor(6.0, shape=(), dtype=float32)

3. Optimized CPU/GPU Support

TensorFlow seamlessly leverages the computational power of GPUs to accelerate operations involving tensors. The framework also supports high-level abstractions like tf.data for efficient data input pipelines.

Here is the Python code:

# GPU usage example
a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
c = tf.matmul(a, b)
print(c)  # Output: matrix product of 'a' and 'b'

4. Multi-Layered API

TensorFlow offers both high-level and low-level APIs, catering to the diverse needs of deep learning practitioners. For beginners, higher-level interfaces like tf.keras and ready-to-use models make it concise and straightforward to get started.

Here is the Python code:

# Keras example
model = tf.keras.Sequential([
    tf.keras.layers.Dense(10, input_shape=(20,)),
    tf.keras.layers.Dense(2)
])
model.compile(optimizer='adam', loss=tf.keras.losses.BinaryCrossentropy())

At the same time, experienced developers can utilize low-level APIs for finer control and customization, such as tf.Module for building custom layers.

Here is the Python code:

# Low-level API example
class MyModule(tf.Module):
    def __init__(self):
        self.dense_layer = tf.keras.layers.Dense(5)
    def __call__(self, x):
        return self.dense_layer(x)

5. Model Serving

TensorFlow is equipped with tools like TensorFlow Serving and TensorFlow Lite that facilitate model deployment. Serving is optimized for distributing machine learning models in production settings, and TensorFlow Lite is designed for deploying models on resource-constrained devices like mobile phones and IoT devices.

6. Extensive Ecosystem and Community

The framework has a rich ecosystem, with ready-to-use pre-trained models available through platforms such as TensorFlow Hub for quick experimentation and prototyping. It also has extensions for incorporating AI/ML into web applications through TensorFlow.js.

Furthermore, TensorFlow has a vast and active community, offering user support, resources, and continually expanding the framework’s capabilities through contributions.

7. Cross-Platform Compatibility

TensorFlow works seamlessly across various operating systems, including Windows, macOS, and Linux, providing uniform behavior and programming interfaces.

8. Scalability

The framework can scale from small experiments on a single machine to handling massive datasets in a distributed manner across multiple machines. This capability makes TensorFlow suitable for both research and production deployments.

TensorFlow represents computational operations via a computational graph, which is a set of nodes (operations) connected by edges (tensors with shared data).

Key Components

• Graph: A collection of nodes and edges that represent data and computations.
• Nodes: Operations that represent computations or mathematical transformations.
• Edges: Tensors that carry data, enabling operations to feed into one another.

Benefits of Computational Graphs

1. Optimization: TensorFlow can optimize the graph by fusing operations, reducing memory usage, and introducing parallelism.
2. Distributed Execution: Graphs can be partitioned for distributed computing.
3. Auto-Differentiation: The graph can efficiently calculate derivatives for use in optimization algorithms.
4. Model Serialization: The graph is the model, making it easier to distribute, exchange, and manage versions.

Code Example: Creating a Simple Computation Graph

Here is the Python code:

import tensorflow as tf

# Define nodes and edges
a = tf.constant(5)
b = tf.constant(2)
c = tf.multiply(a, b)

# Execute the graph
with tf.Session() as sess:
    result = sess.run(c)
    print(result)  # Output: 10

In TensorFlow, everything revolves around a fundamental building block called a tensor. Tensors are multi-dimensional arrays that represent data flow in the computation graph, and they come in different ranks, which can be viewed as their dimensions.

Common Ranks

• Scalar: A single number, such as $5$.
• Vector: A one-dimensional array, such as $[2, 3, 5]$.
• Matrix: A two-dimensional array, like a grid of numbers.
• Higher-Dimensional Tensors: Three or more dimensions, used for more complex data structures.

Code Example: Tensor Ranks

Here is the Python code:

import tensorflow as tf

constant_scalar = tf.constant(5)
constant_vector = tf.constant([2, 3, 5])
constant_matrix = tf.constant([[1, 2], [3, 4]])

with tf.Session() as session:
    print("Rank of the Scalar: ", session.run(tf.rank(constant_scalar)))  # 0
    print("Rank of the Vector: ", session.run(tf.rank(constant_vector)))  # 1
    print("Rank of the Matrix: ", session.run(tf.rank(constant_matrix)))  # 2

Tensor Properties

• Shape: Defines the number of elements along each axis. For instance, a vector of length 3 has the shape [3], and a 2x3 matrix has the shape [2, 3].

• Data Type (dtype): Specifies the type of data within the tensor, such as float32, int64, or string. All elements in a tensor must have the same data type.

  For example, a 3x2 matrix might have a shape of [3, 2] and a data type of float32.

Tensors are the primary means of passing data around in TensorFlow, serving as inputs, intermediate results, and outputs. In essence, they are the mechanism that permits parallel, GPU-accelerated computations and allows TensorFlow to precisely manage and optimize models.

TensorFlow was developed by DeepMind and later open-sourced by Google in 2015. It is is one of the most widely known and used libraries for machine learning and deep learning tasks, offering several unique features and characteristics.

TensorFlow Versus Other ML Libraries

1. Computation Graph: TensorFlow represents computations as a directed graph, which is then executed within sessions. This allows for more flexibility and efficient use of resources, especially for complex operations and advanced techniques like autodifferentiation.

2. Static vs. Dynamic Graphs: In TensorFlow 1.x, the graph is primarily static, meaning it’s defined and then executed as a whole. With TensorFlow 2.x and Eager Execution, dynamic graphs similar to those in PyTorch are better supported, allowing for more intuitive and responsive coding.

3. Integrated Toolset: TensorFlow provides a comprehensive suite of tools for model building, training, and deployment. This includes TensorFlow Serving for serving models in production, TensorFlow Lite for mobile and edge devices, and TensorFlow Hub for sharing trained model components.

4. Advanced Support: TensorFlow is built to handle deep learning tasks at scale. It’s specifically optimized for computations on GPUs and has specialized GPU support modules, like TensorRT for NVIDIA GPUs. Additionally, TensorFlow was designed with distributed computing in mind, making it suitable for distributed training across multiple machines.

5. Deployment in Production: TensorFlow’s model serving capabilities, via TensorFlow Serving, and its compatibility with programming languages like C++ and Java, make it a strong contender for industry-level deployment.

6. Maturity: TensorFlow has been in development for a longer period and has gained a tremendous community following. This longevity contributes to its rich documentation, diverse ecosystem, and extensive support.

TensorFlow primarily runs computations on the CPU or the GPU. More recently, it has extended its support for specialized hardware devices through Tensor Processing Units (TPUs) and Custom Kernels.

CPU and GPU

The out-of-the-box installation of TensorFlow can handle operations on both CPU and GPU. The library provides a unified interface, enabling developers to use high-performance GPU resources without having to deal with the low-level details of hardware-specific optimizations.

CPU Acceleration

While CPUs are generally not as efficient as GPUs for deep learning computations, TensorFlow effectively utilizes their multithreading capabilities. The mkl and eigen backends optimize CPU computations for improved performance.

Transactional Memory ™ and Cache Coherency (CCNUMA)

Modern CPU architectures often rely on mechanisms like Cache Coherency and Transactional Memory. TensorFlow’s design ensures the efficient use of these components, thereby enhancing its performance on CPU-focused computations.

Graph Execution

TensorFlow’s graph execution mode enables the framework to optimize the computation graph before running it. This method is CPU-based and can provide performance benefits by streamlining operations.

GPU Execution

For hardware with GPU support, TensorFlow is equipped with a GPU-focused compute engine. By leveraging the parallel computing capabilities of GPUs, TensorFlow delivers significantly faster performance for various workloads.

The cuDNN library from NVIDIA powers convolutional layers, yielding additional acceleration.

TPU Support

TensorFlow doesn’t natively support TPUs, but TensorFlow Lite and TensorFlow.js offer varying levels of TPU compatibility. The cloud-based Google Colab provides integrated TPU acceleration and is a popular choice for users wanting to harness the power of TPUs.

TensorFlow operates within a graph-based paradigm, where graphs define computational tasks and nodes represent operations or data, leading to ease of parallelism.

A session is the context in which graph operations take place. It’s an abstraction that encapsulates the state of the running graph, such as variable values.

Key Functions

1. Session(): This function initializes a TensorFlow session and sets up the connection to execute a defined computational graph.

2. run(): Used within the session, it causes the operations and nodes to be executed. For example, session.run([node1, node2]) would compute the values of node1 and node2.

3. close(): Once you’re done with the session, it’s important to release resources. Closing the session achieves this.

Code Example: TensorFlow Session

Here is the Tensorflow v1 code:

import tensorflow as tf

# Build a simple graph
a = tf.constant(5)
b = tf.constant(3)
c = tf.multiply(a, b)

# Start a session and run the graph
with tf.Session() as session:
  result = session.run(c)
  
print(result)  # Output: 15

And here is the equivalent Tensorflow v2 code:

import tensorflow as tf

# Build a simple graph
a = tf.constant(5)
b = tf.constant(3)
c = a * b  # Operator overloading

# Start a session and run the graph
result = c.numpy()
print(result)  # Output: 15

In TensorFlow 2, graph operations can often be evaluated outside a session, providing more flexibility.

Role in Computational Graph Execution

• Running Operations: It is through the session that operations in the graph are executed and data is evaluated.
• Managing Resources: The session handles the allocation and release of resources like CPU or GPU memory for tensor objects, minimizing errors and optimizing performance.
• Control Dependencies and Ordening Calculations: It allows you to define dependencies between operations, which can be particularly useful when certain operations should run before others, or for ensuring their ordering.

Multi-GPU Execution

For distributed execution across multiple GPUs, TensorFlow can distribute operations, as defined in the graph, across GPU devices using a multi-GPU session.

Switching Between Graphs

By using multiple sessions, you can work with different computational graphs simultaneously. This can be useful when, for instance, training multiple models independently.

TensorFlow 1.x was primarily imperative in nature, with a focus on defining the entire computation graph before execution. In contrast, TensorFlow 2.x emphasizes a declarative and eager execution approach, aiming to make the framework more intuitive and user-friendly.

Key Changes from TensorFlow 1.x to 2.x

Backward Compatibility

• 1.x: While highly versatile, it sometimes became complicated due to multiple APIs and various ways to achieve tasks.
• 2.x: To streamline and improve user experience, TensorFlow 2.x focused on a more cohesive and unified approach. Many 1.x modules and functionalities are still supported through compatibility modules like tf.compat.v1, ensuring smoother transitions for existing projects.

Eager Execution

• 1.x: By default, TensorFlow 1.x frameworks operated in a define-and-run mode. This meant that user-defined graphs had to be constructed first before data could flow through, making interactive debugging and model building complex.
• 2.x: The introduction of eager execution in TensorFlow 2.x allows for immediate evaluation of operations. It offers a more intuitive, Pythonic experience akin to using NumPy and allows users to see results instantly, enhancing ease of use and debuggability.

Keras Integration

• 1.x: In 1.x, Keras was a high-level API available as a separate library. Users had to install Keras alongside TensorFlow.
• 2.x: Starting from TensorFlow 2.x, Keras is the high-level, official API for model construction within the TensorFlow framework. This integrated Keras provides a unified training pipeline, layer consistencies, and an easy-to-use interface.

Model Building

• 1.x: In TensorFlow 1.x, setting up custom models meant creating and handling operations and placeholders/managing sessions.
• 2.x: The latest iteration excels in simplicity. Models can be built in fewer lines, leveraging the advantages of eager execution and Keras high-level constructs.

API Unification

• 2.x: Offers a consolidated tf.keras package, combining the benefits of the original Keras library with extended support for TensorFlow functionality.

How It Looks in Code

1.x:

# Example of TensorFlow v1 code
import tensorflow as tf

# Construct computational graph
a = tf.constant(2)
b = tf.constant(3)
c = tf.add(a, b)

# Launch the graph in a session
with tf.Session() as sess:
    print(sess.run(c))  # Output: 5

2.x:

# Example of TensorFlow v2 code
import tensorflow as tf

a = tf.constant(2)
b = tf.constant(3)

# Eager execution enables immediate operation
print(a + b)  # Output: tf.Tensor(5, shape=(), dtype=int32)

TensorFlow benefits from its automatic differentiation, a core concept that enables neural network training. TensorFlow achieves this using the technique known as computational graphs.

The Power of Computational Graphs

A computational graph is a series of nodes connected by mathematical operations, visualized as a directed graph. It enables TensorFlow to:

• Optimize Operations: The graph provides a clear sequence of operations, helping TensorFlow manage computational resources.

• Simplify Backpropagation: The graph can be traversed from output to input to efficiently compute gradients, especially in deep models.

Differentiation in TensorFlow

When a computation is defined, TensorFlow automatically maintains a computational graph and calculates the gradients for backward propagation. This process breaks down into two key steps:

1. Graph Definition and Execution: TensorFlow builds a graph that outlines operations and data flow. It then uses a Session (or, in eager mode, an operation) to execute operations within that graph.

2. Automatic Differentiation: TensorFlow tracks the relationships between input and output by employing a mechanism called “auto-diff,” ensuring the graph’s corresponding gradients are calculated accurately.

Eager vs. Graph Mode

TensorFlow offers two modes of operation:

• Eager Execution: Computation is immediate, similar to Python’s default behavior.
• Graph Mode: Computation is deferred, and an entire computational graph is constructed before execution.

Eager mode is more intuitive and offers flexibility, but graph mode is often faster, especially for complex operations.

The following code snippets illustrate both modes:

Eager Mode

Eager mode computes the gradient immediately:

import tensorflow as tf

x = tf.Variable(3.0)
with tf.GradientTape() as tape:
    y = x**2

# The gradient is available immediately after the computation
dy_dx = tape.gradient(y, x)
print(dy_dx)  # Output: 6.0

Graph Mode

In graph mode, a computational graph is built, and the gradient is then computed:

x = tf.Variable(3.0)

# The computation is defined within a function
def compute_y():
    return x**2

# A gradient tape is used within the defined function
with tf.GradientTape(persistent=True) as tape:
    y = compute_y()

# The gradient can be retrieved after the graph is executed
dy_dx = tape.gradient(y, x)
print(dy_dx)  # Output: 6.0

A TensorFlow Placeholder is a type of tensor that holds a place for data to be input during graph execution. This provides a way for feeding data into TensorFlow models, making them dynamic and versatile.

Key Characteristics

• Declaration: Done with tf.placeholder. You specify the data type and, optionally, the tensor’s shape.
• Data Binding: During execution, the data is provided through a dedicated feed mechanism, defined when running operations within a session or when evaluating a specific part of the graph.
• Tensor Life Span: It exists only within the context of a tf.Session and lasts until the session is closed or reset. After that, the placeholder is no longer valid.

Code Example: Declaring and Feeding Placeholders

Here is the Python code:

import tensorflow as tf

# Declaring a placeholder with shape [None, 3]
# The first dimension is left as 'None', indicating it could take inputs of any size
ph = tf.placeholder(tf.float32, shape=[None, 3])

# Simulating input data
input_data = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]

# Creating a graph using the placeholder
result = tf.reduce_sum(ph, axis=1)

# Running the graph within a TensorFlow Session and providing data through the feed_dict mechanism
with tf.Session() as sess:
    # Feeding the placeholder 'ph' with 'input_data' 
    output = sess.run(result, feed_dict={ph: input_data})
    print(output)  # This will print the sums of the three input rows

TensorFlow Lite is a streamlined version of TensorFlow designed for mobile and embedded devices. It offers optimized models and inference through techniques like quantization and model optimization. This results in reduced model sizes and faster predictions, making it ideal for real-time applications.

TensorFlow Lite is often used in scenarios such as mobile image recognition, natural language processing for chatbots, and smart IoT devices that require on-device inference without being continuously connected to the cloud.

Key Features

• Platform Flexibility: TensorFlow Lite supports various hardware backends, including CPUs and GPUs, and accelerators like Google’s Edge TPU.

• Model Optimization Toolkit: TFLite provides tools to optimize, evaluate, and convert models for efficient inference, such as the TFLite Model Maker for task-specific model generation.

• Interpreter Customization: Developers have the option to further optimize inference through techniques like hardware acceleration and selective operator loading with the TFLite interpreter.

TFLite Workflow

1. Model Selection & Training: Start with TensorFlow, train your model, and export it to TensorFlow’s model format (.pb or .SavedModel).

2. Model Conversion: Use TensorFlow’s tflite_convert tool to convert the model into TFLite’s flat buffer format (.tflite).

3. Inference on Device: Incorporate the TFLite model into your mobile or embedded app, and use TFLite tools to make predictions.

Model Optimization Techniques

• Quantization: Reduces the precision of model weights and activations.

• Pruning: Eliminates less important model parameters.

• Model Substructuring: Splits model layers into smaller sub-layers for more efficient execution.

• Post-training Quantization: Quantization techniques applied after model training.

Efficiency Metrics

TFLite provides tools for profiling model efficiency:

• Interpreter: Gives insight into latency and model accuracy.

• Benchmark Tool: Assesses model performance and provides optimization suggestions.

In TensorFlow, a Variable $V$ is a tensor that’s mutable and adaptable, designed to hold and update state throughout the graph’s execution.

Key Characteristics

• Mutability: Allows the tensor value to be assigned and reassigned.

• Persistence: Withholds its value across graph executions.

• Automatic Partial Derivatives Management:

  • Adjusting variances in computations for better outcomes.
  • Particularly beneficial in optimization algorithms.

• Memory Location: Variable data is saved in the memory.

• Initialization Requirement: Should be initialized, and initial value is expected to be set.

• Resource Handling Caveats: Variables should be managed using context managers like tf.Session or tf.GradientTape.

Code Example: Defining a TensorFlow Variable

Here is the Python code:

import tensorflow as tf

# Variable Initialization
initial_value = tf.constant(3.14)
V = tf.Variable(initial_value)

# Assignment Operator
new_value = tf.constant(2.71)
assignment_op = tf.assign(V, new_value)

# Variable Initialization
init_op = tf.global_variables_initializer()
with tf.Session() as session:
    session.run(init_op)
    print(session.run(V))  # Output: 3.14
    session.run(assignment_op)
    print(session.run(V))  # Output: 2.71

TensorFlow supports several data types that are compatible with various hardware configurations and numerical precision needs. This ensures flexibility in designing models and ensures compatibility with different backend devices.

Common Data Types

Data Types Compatible with CPU and GPU

• Boolean: True or False values.
• Integer: Whole numbers.
  • int8, int16, int32, int64: Signed integers of different lengths.
  • uint8, uint16: Unsigned integers of different lengths.
• Floating-Point: Decimals with different precisions.
  • float16: Half-precision floating point. Not compatible with CPU.
  • float32: Standard single-precision floating point.
  • float64: Double-precision floating point.

Data Types for GPU Operations

• Complex Numbers: Numbers with both real and imaginary parts.
  • complex64: Single-precision complex numbers.
  • complex128: Double-precision complex numbers.

Specialized Data Types

These data types cater to specific use-cases and hardware configurations.

Quantized Data Types

• qint8, quint8: 8-bit integers with or without sign. Used for quantized neural networks.

String Data Type

• string: Represents a string of variable length. Primarily used with tf.data to represent tensor elements that are sequences or subsets of text data.

Optional Data Types for Specialized Hardware

• Bfloat16: For brain floating point, a specialized 16-bit float used to reduce memory requirements and speed up large network training, particularly on TPU devices.

• Dtype (TensorFlow 2.1+): An object representing data type.

Choosing the Right Data Type

The data type selection for your model should consider the following:

• Numerical Precision: Choose the smallest data type that maintains necessary precision. This reduces memory and computational requirements.
• Hardware Compatibility: Ensure the selected data type is compatible with the hardware for model efficiency.
• Specific Use-Cases: Tailor the data type choice to the requirements of your tasks, especially for neural network quantization.

Building a Neural Network in Tensorflow generally entails the following steps:

1. Initialize the Graph
2. Define Network Architecture
3. Set Hyperparameters
4. Initialize Variables
5. Train the Model

Importing Libraries

Before diving into the code, make sure you have the necessary libraries installed. If you’re using Google Colab or similar platforms, Tensorflow is generally already available.

!pip install tensorflow

Here’s the TensorFlow 2.x code:

Initializing the Graph

import tensorflow as tf

# Clear any previous graph
tf.keras.backend.clear_session()

# Set random seed for reproducibility
tf.random.set_seed(42)
# np.random.seed(42)

# Instantiate model
model = tf.keras.Sequential()

Defining Network Architecture

# Add layers to the model
model.add(tf.keras.layers.Dense(128, input_shape=(784,), activation='relu'))
model.add(tf.keras.layers.Dropout(0.2))
model.add(tf.keras.layers.Dense(10, activation='softmax'))

Set Hyperparameters

# Compile the model
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Summarize and visualize the model architecture
model.summary()
# tf.keras.utils.plot_model(model, to_file='model.png', show_shapes=True)

Initialize Variables

In Tensorflow 2.x, you typically don’t need to initialize variables explicitly.

Train the Model

# Load and preprocess datasets
mnist = tf.keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
train_images, test_images = train_images / 255.0, test_images / 255.0

# Train the model
model.fit(train_images, train_labels, epochs=5, validation_data=(test_images, test_labels))

Compiling a model in TensorFlow refers to the configuration of details like loss functions, optimizers, and metrics. This sets the groundwork for model training.

Steps in Compiling a Model

1. Optimizer Selection: Choose the algorithm to update model parameters, aiming to minimize the loss function. Common choices include Stochastic Gradient Descent (SGD) and its variants, e.g., Adam.

2. Loss Function Designation: Quantify the difference between predicted and actual values. The type of ML task (e.g., regression or classification) usually dictates the appropriate loss function.

3. Performance Metric Definition: Metrics such as accuracy or area under the ROC curve help evaluate the model during training and testing.

Code Example: Compiling a Model

Here is the Python code:

# Import TensorFlow and any dataset/input data that's needed
import tensorflow as tf
from tensorflow import keras

# Load dataset
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()

# Data preprocessing, e.g., scaling
X_train = X_train/255
X_test = X_test/255

# Build the model
model = keras.Sequential([
    keras.layers.Flatten(input_shape=(28, 28)),
    keras.layers.Dense(128, activation='relu'),
    keras.layers.Dense(10, activation='softmax')
])

# Compile the model
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

In this example:

• The Adam optimizer, chosen for its general effectiveness, will tweak model parameters.
• The sparse categorical cross-entropy loss function is set up to suit a multi-class classification problem, e.g., recognizing digits in the MNIST dataset.
• The accuracy metric will be used to evaluate model performance.