Bash: Quick File Processing Examples

1. Find and process all files in a directory and its subdirectories

#!/bin/bash

# Find and process all .txt files in the current directory and its subdirectories
# The -print0 option and 'read -r -d '' ' are used to handle filenames with spaces
find . -type f -name "*.txt" -print0 | while read -r -d '' file; do
    echo "Processing file: $file"
done

2. Process all files in a single directory

#!/bin/bash

# Process all files in a specific directory
# The -1 option ensures one file per line, and IFS= read -r ensures correct processing
IFS=
ls -1 /path/to/my/directory | while read -r file; do
    echo "Processing file: $file"
done

3. Read a file line by line

#!/bin/bash

# Process a file one line at a time
while read -r line; do
    echo "Processing line: $line"
done < "my_file.txt"

4. Read a full CSV file into an array of arrays

This example reads each line of a CSV file and stores it as a separate array within a master array.

#!/bin/bash

declare -a full_csv_data
while read -r line; do
    IFS=',' read -r -a csv_line_array <<< "$line"
    full_csv_data+=( "$(declare -p csv_line_array)" )
done < "data.csv"

# Print the contents of the array of arrays
for arr_string in "${full_csv_data[@]}"; do
    eval "$arr_string"
    echo "Record: ${csv_line_array[0]} ${csv_line_array[1]}"
done