NumPy Mastery: Arrays, Operations & Real Data Projects

What is a Jupyter Notebook

A Jupyter Notebook is a special file (.ipynb) that lets you write code, run it, and see output in one place. Unlike Python files, notebooks support markdown text, equations, images, and charts—making them ideal for data visualization and analysis workflows.

Three Ways to Use Notebooks

You can use Jupyter Notebooks via: (1) Anaconda Distribution with Jupyter Notebook installed locally, (2) VS Code with Jupyter extension, or (3) Google Colab (cloud-based with free GPU support). Each has trade-offs between ease of setup and available resources.

Notebook environments ranked by setup complexity and features

Key Notebook Settings

Enable auto-completion in settings to get function suggestions while typing. Enable auto-close brackets to avoid syntax errors. These settings are essential because NumPy has thousands of functions you won't remember.

NumPy's Evolution

NumPy was created in 2005 by Travis Oliphant by combining two earlier libraries: Numeric (created 1995) and Numarray. This unified library brought array support to Python with dramatically improved performance.

NumPy development timeline

Why NumPy is 50x Faster Than Python Lists

NumPy is written in C and compiled, while Python is interpreted. NumPy arrays are stored contiguously in memory and optimized for mathematical operations, making them dramatically faster for numerical computations than Python's native list data structure.

50x

NumPy arrays faster than Python lists

Performance advantage due to C implementation and compilation

Data Type Homogeneity

Python lists can store mixed data types (integers, floats, strings) in one collection. NumPy arrays enforce a single data type across all elements—if you mix types, NumPy converts everything to a common type (usually string or float).

Data type handling: lists vs arrays

Performance & Memory Efficiency

NumPy arrays are much faster (up to 50x) and use significantly less memory than Python lists because they store homogeneous data contiguously and leverage C-level optimizations.

Performance comparison: speed and memory efficiency

Vector Operations

With Python lists, you must manually loop through elements to perform operations. NumPy arrays support vectorized operations—apply the same operation to all elements at once using broadcasting, eliminating the need for explicit loops.

Creating Arrays from Lists

Use np.array() to convert a Python list into a NumPy array. The resulting array will have a single data type; mixed types are automatically converted to a common type.

Generating Arrays with np.arange()

np.arange(start, stop, step) generates evenly spaced numbers similar to Python's range(). Note: the stop value is exclusive, so use stop+1 if you want to include it.

np.arange(1, 11)

Generates 1 through 10

Stop value is exclusive; use 11 to include 10

Generating Zeros and Ones

np.zeros(shape) creates an array filled with zeros; np.ones(shape) creates an array filled with ones. For 2D arrays, pass a tuple like (6, 5) for 6 rows and 5 columns.

Linear Spacing with np.linspace()

np.linspace(start, stop, num) generates exactly 'num' evenly spaced values between start and stop (inclusive). Unlike arange, the stop value is included, and spacing is uniform across all intervals.

np.linspace(1, 5, 5)

Generates [1, 2, 3, 4, 5] with equal spacing

Useful for creating evenly distributed sample points

Random Number Generation

np.random.rand(n) generates n random floats between 0 and 1 (normalized). np.random.randn(n) generates n random floats between -3 and 3 (standardized). np.random.randint(low, high, size) generates random integers in the range [low, high).

Key Array Attributes

arr.shape returns dimensions (e.g., (3, 3) for 3 rows, 3 columns). arr.size returns total element count. arr.dtype returns the data type. These are properties, not methods—no parentheses needed.

Statistical Methods

arr.min() and arr.max() find minimum and maximum values. arr.sum() sums all elements. arr.mean() calculates average. arr.std() computes standard deviation. Use axis parameter (0 for columns, 1 for rows) to apply along specific dimensions.

Index-Finding Methods

arr.argmax() returns the index of the maximum element. arr.argmin() returns the index of the minimum element. These are useful for finding positions rather than values.

Reshaping Arrays

arr.reshape(new_shape) reorganizes array elements into a new shape without changing the total element count. For example, a 30-element array can be reshaped to (6, 5) but not (5, 5). The product of dimensions must match.

Reshape requires matching total element count

Resizing Arrays

arr.resize(new_shape) modifies the array in-place to a new shape. Unlike reshape, resize can change the total number of elements (padding with zeros or truncating as needed).

Vector Indexing

Access individual elements using zero-based indexing: arr[0] gets the first element, arr[5] gets the sixth. Negative indices count from the end: arr[-1] is the last element.

Vector Slicing

arr[start:stop:step] extracts a portion of the array. The stop index is exclusive. Omit start to begin at 0, omit stop to go to the end, omit step for step=1. Example: arr[1:5] gets elements at indices 1, 2, 3, 4.

arr[1:5:2]

Gets indices 1 and 3 (step of 2)

Slicing syntax: [start:stop:step]

Selecting Single Elements

For a 2D array, arr[row, col] selects the element at that position. Row and column indices are zero-based. Example: arr[0, 4] gets the element in the first row, fifth column.

Selecting Rows and Columns

arr[row_index, :] selects an entire row. arr[:, col_index] selects an entire column. The colon means 'all elements in that dimension'.

Slicing Matrix Portions

arr[row_start:row_stop, col_start:col_stop] extracts a rectangular region. Example: arr[0:3, 1:3] gets rows 0–2 and columns 1–2. Both dimensions use the same slicing rules as vectors.

Filtering with Boolean Conditions

Create a boolean array by applying a condition: bool_index = arr % 2 == 0 produces True/False for each element. Then use arr[bool_index] to extract only elements where the condition is True.

Boolean Array Requirements

The boolean array must have the same shape as the original array. Each True value keeps the corresponding element; each False value removes it. This is powerful for conditional filtering without explicit loops.

Element-Wise Arithmetic

NumPy performs arithmetic operations element-by-element: arr1 + arr2 adds corresponding elements, arr1 * arr2 multiplies them, etc. Both arrays must have the same shape, or broadcasting rules apply.

Broadcasting Concept

Broadcasting allows operations between arrays of different shapes. If one array is smaller, NumPy automatically expands it to match the larger array's shape. Example: arr + 10 adds 10 to every element without explicit loops.

arr + 10

Broadcasts scalar to all elements

Broadcasting eliminates the need for manual loops

Deep Copy vs Shallow Copy

b = a creates a shallow copy—both variables point to the same data, so changes to b affect a. b = a.copy() creates a deep copy—b is independent, and changes don't affect a. Slicing creates a separate copy in memory.

Memory sharing vs independence

Matrix Multiplication

Use arr1 @ arr2 (or np.dot(arr1, arr2)) for true matrix multiplication, not element-wise multiplication. In matrix multiplication, each element of the result is the dot product of a row from the first matrix and a column from the second.

Matrix Transpose

arr.T flips rows and columns: rows become columns and columns become rows. For a (3, 4) matrix, transpose produces a (4, 3) matrix. Useful for reshaping data for specific algorithms.

Vertical & Horizontal Stacking

np.vstack((arr1, arr2)) stacks arrays vertically (row-wise). np.hstack((arr1, arr2)) stacks horizontally (column-wise). Both require compatible shapes. Use tuples as arguments.

Column Stacking

np.column_stack((arr1, arr2)) treats 1D arrays as columns and combines them side-by-side into a 2D array. Useful for building data matrices from separate feature vectors.

Splitting Arrays

np.hsplit(arr, n) splits horizontally into n equal parts. np.vsplit(arr, n) splits vertically into n equal parts. The array dimensions must be evenly divisible by n.

Sudoku Rules

A valid Sudoku has 9 rows, 9 columns, and nine 3×3 blocks. Each row, column, and block must contain the digits 1–9 exactly once. The sum of each row, column, and block must equal 45 (sum of 1–9).

45

Sum of each valid row, column, or 3×3 block

Sudoku validation uses sum checks

Validating Rows and Columns

Use np.sum(sudoku, axis=1) to sum each row and check if all equal 45. Use np.sum(sudoku, axis=0) to sum each column. If any sum ≠ 45, the Sudoku is invalid.

Validating 3×3 Blocks

Extract each 3×3 block using slicing: sudoku[i:i+3, j:j+3] where i and j iterate 0, 3, 6. Sum each block and verify it equals 45. Use nested loops with step=3 to iterate through all nine blocks.

Array Shape & Dimensions

Use arr.shape to get dimensions. For a student dataset with 5 students and 3 subjects (age, math, science), shape is (5, 3)—5 rows, 3 columns.

Calculating Averages

np.mean(arr[:, col_index]) calculates the average of a specific column. np.mean(arr, axis=0) calculates column-wise averages; axis=1 calculates row-wise averages.

Extracting Specific Columns

Use arr[:, col_index] to extract an entire column. Example: arr[:, 1] gets all math marks. Use boolean indexing to filter: arr[arr[:, 1] > 90] gets rows where math marks exceed 90.

Finding Maximum Values

np.max(arr[:, col_index]) finds the highest value in a column. Example: np.max(arr[:, 2]) finds the highest science mark.

Conditional Filtering with Multiple Conditions

Combine conditions using & (and) operator: (arr[:, 1] > 90) & (arr[:, 2] > 80) filters rows where both math > 90 AND science > 80. Wrap each condition in parentheses.

Modifying Array Values

Use arr[:, col_index] += value to add to all elements in a column. Use arr[condition] = value to replace elements matching a condition. Example: arr[arr[:, 2] < 75, 2] = 0 sets science marks below 75 to zero.