BashPitfalls - Greg's Wiki

This page is a compilation of common mistakes made by bash users. Each example is flawed in some way. Yes, it would be great if you could just treat the output of ls or find as a list of filenames and iterate over it. This entire approach is fatally flawed, and there is no trick that can make it work. You must use an entirely different approach. If a filename contains whitespace, it undergoes WordSplitting. Assuming we have a file named 01 - Don't Eat the Yellow Snow.mp3 in the current directory, the for loop will iterate over each word in the resulting file name: 01, -, Don't, Eat, etc. If a filename contains glob characters, it undergoes filename expansion ("globbing"). If ls produces any output containing a * character, the word containing it will become recognized as a pattern and substituted with a list of all filenames that match it. If the command substitution returns multiple filenames, there is no way to tell where the first one ends and the second one begins. Pathnames may contain any character except NUL. Depending on which platform you're on, which arguments you used (or didn't use), and whether its standard output is pointing to a terminal or not, ls may randomly decide to replace certain characters in a filename with "?", or simply not print them at all. Never try to parse the output of ls. It's an external command whose output is intended specifically to be read by a human, not parsed by a script. That may seem desirable since ls adds a newline, but if the last filename in the list ends with a newline, ... or $() will remove that one also. In the ls examples, if the first filename starts with a hyphen, it may lead to pitfall #3. This causes the entire output of ls to be treated as a single word. Instead of iterating over each file name, the loop will only execute once, assigning to f a string with all the filenames rammed together. Nor can you simply change IFS to a newline.