Loop Through Files in Directory in Bash

The short answer

To create a Bash script that loops through files in a specified directory and prints their path, you can start by creating a new file using the [.inline-code]touch[.inline-code] command:

 $ touch iterate.sh

And copy-paste the following script into it, that will:

1. Check if the target is a valid directory or exit with a return value of 1, which in Unix translates to “something went wrong”.
2. Create a [.inline-code]for[.inline-code] loop that iterates over each file of the target directory.
3. Check if the current file is a regular file.
4. Write the file path to the standard output using the [.inline-code]echo[.inline-code] command.

 #!/bin/bash

# Define the target directory
directory="/path/to/directory"

# Check if the target is not a directory
if [ ! -d "$directory" ]; then
  exit 1
fi

# Loop through files in the target directory
for file in "$directory"/*; do
  if [ -f "$file" ]; then
    echo "$file"
  fi
done

To be able to run the script, you then have to give it execution permission using the [.inline-code]chmod[.inline-code] command:

 $ chmod +x iterate.sh

Finally, you can run the script using the following command:

 $ ./iterate.sh

If you want to learn more advanced formatting options, you can read our article on the printf utility.

If you're new to Unix permissions, you can also read our other article on the [.inline-code]chmod +x[.inline-code] command.

[#using-warps-ai]Using Warp's AI to quickly retrieve these steps [#using-warps-ai]

If you’re using Warp as your terminal, you can easily retrieve an approximation of the above script using Warp's AI feature:

‍

For that:

1. Click on the bolt icon on the top right of the terminal
2. Type in your question; for example: "Write a Bash script that prints the name of files in a directory".
3. Press [.inline-code]ENTER[.inline-code] to generate an answer.

As with any AI-powered tool, use it as a starting point and not a final answer.

Including and excluding files

[#including-hidden-files]Including hidden files [#including-hidden-files]

Since, by default, hidden files are not listed, you can add the following [.inline-code]shopt[.inline-code] command (short for shell options) to the script, right before the [.inline-code]for[.inline-code] loop, to enable Bash to include dot files in the result of the filename expansion.

 #!/bin/bash

directory="/path/to/directory"

shopt -s dotglob

for file in "$directory"/*; do
  if [ -f "$file" ]; then
    echo "$file"
  fi
done

[#including-files-based-on-a-pattern]Including files based on a pattern[#including-files-based-on-a-pattern]

To only include files that match a specific pattern (e.g. JavaScript or JSON files), you can use a [.inline-code]pattern[.inline-code] variable that contains a globbing expression as follows:

 #!/bin/bash

directory="/path/to/directory"
pattern="*.json"

for file in "$directory"/$pattern; do
  if [ -f "$file" ]; then
    echo "$file"
  fi
done

[#excluding-files-based-on-a-pattern]Excluding files based on a pattern[#excluding-files-based-on-a-pattern]

To exclude files that match a specific pattern, you can use a combination of a [.inline-code]while[.inline-code] loop, and the [.inline-code]find[.inline-code] and [.inline-code]read[.inline-code] commands, where:

• The [.inline-code]find[.inline-code] command will return a string of characters containing the paths of the regular files that do not match the specified pattern in the target directory, each followed by a trailing null character.
• The [.inline-code]read[.inline-code] command will split the data sent by the [.inline-code]find[.inline-code] command through the pipe into multiple entries using the [.inline-code]-d[.inline-code] option flag, so that they can be written to the standard output by the [.inline-code]echo[.inline-code] command.

 #!/bin/bash

directory="/path/to/directory"
pattern="*.json"

find "$directory" -type f ! -name "$pattern" -print0 -maxdepth 1 | while read -r -d '' file; do
  echo "$file"
done

Note that you can make this script recursively descend in the sub-directories of the target directory by changing the value of the [.inline-code]-maxdepth[.inline-code] option flag.

[#looping-through-files-recursively] Looping through files recursively [#looping-through-files-recursively]

To loop through files in a directory and its subdirectories, you can write a recursive function that takes as initial argument the path of the target directory, iterates through each entry of the current directory, and invokes itself whenever that entry is a directory.

 #!/bin/bash

directory="/path/to/directory"

function iterate() {
  local dir="$1"

  for file in "$dir"/*; do
    if [ -f "$file" ]; then
      echo "$file"
    fi

    if [ -d "$file" ]; then
      iterate "$file"
    fi
  done
}

iterate "$directory"

[#looping-through-lines-in-a-file]Looping through lines in a file[#looping-through-lines-in-a-file]

To loop through the lines of a regular file, you can use the [.inline-code]read[.inline-code] command, which is used to read a line from the standard input and split it into fields, where the [.inline-code]-r[.inline-code] option flag is used to prevent backslashes to escape characters.

 #!/bin/bash

file="/path/to/file"

while IFS= read -r line; do
  echo "$line"
done < "$file"

‍

Note that, setting the IFS (Internal Field Separator) variable to an empty value ensures that leading and trailing whitespace characters are preserved in the input line.

Common use cases

Here are a few examples of common loops you may find useful when iterating over files.

[#example-changing-file-extensions]Changing file extensions[#example-changing-file-extensions]

This loop will change the file extension of JavaScript files from [.inline-code].js[.inline-code] to [.inline-code].jsx[.inline-code].

 #!/bin/bash

directory="/path/to/directory"

for file in "$directory"/*.js; do
  if [ -f "$file" ]; then
    mv "$file" "${file%.js}.jsx"
  fi
done

[#example-cleaning-up-old-files]Cleaning up old files[#example-cleaning-up-old-files]

This loop will delete all the files that are older than 30 days.

 #!/bin/bash

directory="/path/to/directory"

find "$directory" -type f -mtime +30 -print0 | while read -r -d '' file; do
  rm "$file"
done

[#example-extracting-data-from-a-csv-file]Extracting data from a CSV file[#example-extracting-data-from-a-csv-file]

This loop will extract the email addresses contained in a CSV file and write them into a new file named [.inline-code]emails.txt[.inline-code].

#!/bin/bash

file="/path/to/file"

while IFS=',' read -r first_name last_name email_address phone_number; do
  echo "$email_address" >> emails.txt
done < "$file"

[#looping-through-files-in-python]Looping through files in Python[#looping-through-files-in-python]

To re-implement the introduction script in Python, you can write the following code into a new file named [.inline-code]iterate.py[.inline-code]:

import os

directory = '/path/to/directory'

if not os.path.isdir(directory):
    exit(1)

for filename in os.listdir(directory):
    filepath = os.path.join(directory, filename)

    if os.path.isfile(filepath):
        print(f"{filepath}")
        

And run it using the [.inline-code]python[.inline-code] CLI as follows:

$ python3 iterate.py