Moving files and folders into hashed subfolders
The Performance Zone is presented by AppDynamics. Scalability and better performance are constant concerns for the developer and operations manager. Try AppDynamics' fully-featured performance tool for Java, .NET, PHP, & Node.js.
The following will move a series of files into subfolders. It hashes the file names, building a two character, two folder deep hierarchy to split the files, e.g. a1/b2. The motivation for this is to split 500,000 folders into a manageable hierarchy, to avoid file system limits – in NTFS this is quite slow, and well over the 64,000 limit in ext4.
vi f for f in `ls pacer/`; do echo -n $f | md5sum | sed 's`\(..\)\(..\).*`\1/\2/`' | tr -d '\n'; echo $f; done; ./f > t mkdir ../pacer2 ./t | sed 's`\(..\)/\(..\)/\(.*\)`mkdir pacer2/\1\nmkdir pacer2/\1/\2\nmv pacer/\3 pacer2/\1/\2/\3`' > dirs ./dirs