Programing

find로 반환 된 파일 이름을 반복하는 방법은 무엇입니까?

lottogame 2020. 5. 17. 10:27
반응형

find로 반환 된 파일 이름을 반복하는 방법은 무엇입니까?


x=$(find . -name "*.txt")
echo $x

Bash 셸에서 위의 코드를 실행하면 목록이 아닌 공백으로 구분 된 여러 파일 이름이 포함 된 문자열이 나타납니다.

물론 목록을 얻기 위해 공백으로 더 분리 할 수는 있지만 더 좋은 방법이 있다고 확신합니다.

find명령 결과를 반복하는 가장 좋은 방법은 무엇 입니까?


TL; DR : 가장 정확한 답변을 위해 여기에 온다면 내 개인적인 취향을 원할 것 find . -name '*.txt' -exec process {} \;입니다 (이 글의 하단 참조). 시간이 있다면 나머지 부분을 읽고 여러 가지 다른 방법과 대부분의 문제를 확인하십시오.


전체 답변 :

가장 좋은 방법은 수행하려는 작업에 따라 다르지만 몇 가지 옵션이 있습니다. 하위 트리의 파일이나 폴더에 이름에 공백이없는 경우 파일을 반복 할 수 있습니다.

for i in $x; do # Not recommended, will break on whitespace
    process "$i"
done

조금 더 나은 임시 변수를 잘라내십시오 x.

for i in $(find -name \*.txt); do # Not recommended, will break on whitespace
    process "$i"
done

당신이 할 수있을 때 glob하는 것이 훨씬 좋습니다. 현재 디렉토리의 파일에 대한 공백 안전 :

for i in *.txt; do # Whitespace-safe but not recursive.
    process "$i"
done

globstar옵션 을 활성화하면 이 디렉토리와 모든 하위 디렉토리에서 일치하는 모든 파일을 가져올 수 있습니다.

# Make sure globstar is enabled
shopt -s globstar
for i in **/*.txt; do # Whitespace-safe and recursive
    process "$i"
done

예를 들어 파일 이름이 이미 파일에있는 경우 read다음 을 사용해야합니다 .

# IFS= makes sure it doesn't trim leading and trailing whitespace
# -r prevents interpretation of \ escapes.
while IFS= read -r line; do # Whitespace-safe EXCEPT newlines
    process "$line"
done < filename

readfind구분 기호를 적절하게 설정하면 다음 과 함께 안전하게 사용할 수 있습니다 .

find . -name '*.txt' -print0 | 
    while IFS= read -r -d '' line; do 
        process $line
    done

보다 복잡한 검색의 경우 옵션 또는 다음과 find함께을 사용하는 것이 -exec좋습니다 -print0 | xargs -0.

# execute `process` once for each file
find . -name \*.txt -exec process {} \;

# execute `process` once with all the files as arguments*:
find . -name \*.txt -exec process {} +

# using xargs*
find . -name \*.txt -print0 | xargs -0 process

# using xargs with arguments after each filename (implies one run per filename)
find . -name \*.txt -print0 | xargs -0 -I{} process {} argument

find또한 -execdir대신을 사용하여 명령을 실행하기 전에 각 파일의 디렉토리에 CD를 넣을 -exec수 있으며 -ok대신 -exec(또는 -okdir대신 )을 사용하여 대화식 (각 파일에 대해 명령을 실행하기 전에 프롬프트)으로 만들 수 있습니다 -execdir.

* : 기술적으로 findand xargs(기본적으로)는 모든 파일을 처리하는 데 걸리는 횟수만큼 명령 줄에 입력 할 수있는 인수 수만큼 명령을 실행합니다. 실제로 파일 수가 매우 많지 않은 한 중요하지 않으며 길이를 초과하지만 동일한 명령 줄에 모두 필요한 경우 SOL 은 다른 방법을 찾습니다.


find . -name "*.txt"|while read fname; do
  echo "$fname"
done

참고 : bmargulies로 표시되는 이 방법 (두 번째) 방법은 파일 / 폴더 이름의 공백과 함께 사용하는 것이 안전합니다.

파일 / 폴더 이름에 개행 문자가 포함되도록하기 위해 다음 -execfind같은 조건을 사용해야 합니다.

find . -name '*.txt' -exec echo "{}" \;

{}발견 된 항목에 대한 자리 표시 자이며,이 \;종료하는 데 사용되는 -exec술어를.

그리고 완전성을 위해 또 다른 변형을 추가하겠습니다. 다목적 성을 위해 * nix 방법을 좋아해야합니다.

find . -name '*.txt' -print0|xargs -0 -n 1 echo

이것은 인쇄 된 항목을 \0파일 또는 폴더 이름의 파일 시스템에서 허용되지 않는 문자로 분리 하므로 모든 기초를 다루어야합니다. xargs하나씩 하나씩 집어 들고 ...


무엇을하든 루프를 사용하지 마십시오for .

# Don't do this
for file in $(find . -name "*.txt")
do
    …code using "$file"
done

세 가지 이유 :

  • For the for loop to even start, the find must run to completion.
  • If a file name has any whitespace (including space, tab or newline) in it, it will be treated as two separate names.
  • Although now unlikely, you can overrun your command line buffer. Imagine if your command line buffer holds 32KB, and your for loop returns 40KB of text. That last 8KB will be dropped right off your for loop and you'll never know it.

Always use a while read construct:

find . -name "*.txt" -print0 | while read -d $'\0' file
do
    …code using "$file"
done

The loop will execute while the find command is executing. Plus, this command will work even if a file name is returned with whitespace in it. And, you won't overflow your command line buffer.

The -print0 will use the NULL as a file separator instead of a newline and the -d $'\0' will use NULL as the separator while reading.


Filenames can include spaces and even control characters. Spaces are (default) delimiters for shell expansion in bash and as a result of that x=$(find . -name "*.txt") from the question is not recommended at all. If find gets a filename with spaces e.g. "the file.txt" you will get 2 separated strings for processing, if you process x in a loop. You can improve this by changing delimiter (bash IFS Variable) e.g. to \r\n, but filenames can include control characters - so this is not a (completely) safe method.

From my point of view, there are 2 recommended (and safe) patterns for processing files:

1. Use for loop & filename expansion:

for file in ./*.txt; do
    [[ ! -e $file ]] && continue  # continue, if file does not exist
    # single filename is in $file
    echo "$file"
    # your code here
done

2. Use find-read-while & process substitution

while IFS= read -r -d '' file; do
    # single filename is in $file
    echo "$file"
    # your code here
done < <(find . -name "*.txt" -print0)

Remarks

on Pattern 1:

  1. bash returns the search pattern ("*.txt") if no matching file is found - so the extra line "continue, if file does not exist" is needed. see Bash Manual, Filename Expansion
  2. shell option nullglob can be used to avoid this extra line.
  3. "If the failglob shell option is set, and no matches are found, an error message is printed and the command is not executed." (from Bash Manual above)
  4. shell option globstar: "If set, the pattern ‘**’ used in a filename expansion context will match all files and zero or more directories and subdirectories. If the pattern is followed by a ‘/’, only directories and subdirectories match." see Bash Manual, Shopt Builtin
  5. other options for filename expansion: extglob, nocaseglob, dotglob & shell variable GLOBIGNORE

on Pattern 2:

  1. filenames can contain blanks, tabs, spaces, newlines, ... to process filenames in a safe way, find with -print0 is used: filename is printed with all control characters & terminated with NUL. see also Gnu Findutils Manpage, Unsafe File Name Handling, safe File Name Handling, unusual characters in filenames. See David A. Wheeler below for detailed discussion of this topic.

  2. There are some possible patterns to process find results in a while loop. Others (kevin, David W.) have shown how to do this using pipes:

    files_found=1 find . -name "*.txt" -print0 | while IFS= read -r -d '' file; do # single filename in $file echo "$file" files_found=0 # not working example # your code here done [[ $files_found -eq 0 ]] && echo "files found" || echo "no files found"
    When you try this piece of code, you will see, that it does not work: files_found is always "true" & the code will always echo "no files found". Reason is: each command of a pipeline is executed in a separate subshell, so the changed variable inside the loop (separate subshell) does not change the variable in the main shell script. This is why I recommend using process substitution as the "better", more useful, more general pattern.
    See I set variables in a loop that's in a pipeline. Why do they disappear... (from Greg's Bash FAQ) for a detailed discussion on this topic.

Additional References & Sources:


# Doesn't handle whitespace
for x in `find . -name "*.txt" -print`; do
  process_one $x
done

or

# Handles whitespace and newlines
find . -name "*.txt" -print0 | xargs -0 -n 1 process_one

(Updated to include @Socowi's execellent speed improvement)

With any $SHELL that supports it (dash/zsh/bash...):

find . -name "*.txt" -exec $SHELL -c '
    for i in "$@" ; do
        echo "$i"
    done
' {} +

Done.


Original answer (shorter, but slower):

find . -name "*.txt" -exec $SHELL -c '
    echo "$0"
' {} \;

You can store your find output in array if you wish to use the output later as:

array=($(find . -name "*.txt"))

Now to print the each element in new line, you can either use for loop iterating to all the elements of array, or you can use printf statement.

for i in ${array[@]};do echo $i; done

or

printf '%s\n' "${array[@]}"

You can also use:

for file in "`find . -name "*.txt"`"; do echo "$file"; done

This will print each filename in newline

To only print the find output in list form, you can use either of the following:

find . -name "*.txt" -print 2>/dev/null

or

find . -name "*.txt" -print | grep -v 'Permission denied'

This will remove error messages and only give the filename as output in new line.

If you wish to do something with the filenames, storing it in array is good, else there is no need to consume that space and you can directly print the output from find.


If you can assume the file names don't contain newlines, you can read the output of find into a Bash array using the following command:

readarray -t x < <(find . -name '*.txt')

Note:

  • -t causes readarray to strip newlines.
  • It won't work if readarray is in a pipe, hence the process substitution.
  • readarray is available since Bash 4.

Bash 4.4 and up also supports the -d parameter for specifying the delimiter. Using the null character, instead of newline, to delimit the file names works also in the rare case that the file names contain newlines:

readarray -d '' x < <(find . -name '*.txt' -print0)

readarray can also be invoked as mapfile with the same options.

Reference: https://mywiki.wooledge.org/BashFAQ/005#Loading_lines_from_a_file_or_stream


I like to use find which is first assigned to variable and IFS switched to new line as follow:

FilesFound=$(find . -name "*.txt")

IFSbkp="$IFS"
IFS=$'\n'
counter=1;
for file in $FilesFound; do
    echo "${counter}: ${file}"
    let counter++;
done
IFS="$IFSbkp"

Just in case you would like to repeat more actions on the same set of DATA and find is very slow on your server (I/0 high utilization)


You can put the filenames returned by find into an array like this:

array=()
while IFS=  read -r -d ''; do
    array+=("$REPLY")
done < <(find . -name '*.txt' -print0)

Now you can just loop through the array to access individual items and do whatever you want with them.

Note: It's white space safe.


based on other answers and comment of @phk, using fd #3:
(which still allows to use stdin inside the loop)

while IFS= read -r f <&3; do
    echo "$f"

done 3< <(find . -iname "*filename*")

find <path> -xdev -type f -name *.txt -exec ls -l {} \;

This will list the files and give details about attributes.


How about if you use grep instead of find?

ls | grep .txt$ > out.txt

Now you can read this file and the filenames are in the form of a list.

참고URL : https://stackoverflow.com/questions/9612090/how-to-loop-through-file-names-returned-by-find

반응형