{{ !articles[0].partner.isSponsoringArticle ? "Platinum" : "Portal" }} Partner
architects,high-perf,performance,tips and tricks

Moving files and folders into hashed subfolders

The following will move a series of files into subfolders. It hashes the file names, building a two character, two folder deep hierarchy to split the files, e.g. a1/b2. The motivation for this is to split 500,000 folders into a manageable hierarchy, to avoid file system limits – in NTFS this is quite slow, and well over the 64,000 limit in ext4.

vi f
for f in `ls pacer/`; do echo -n $f | md5sum | sed 's`\(..\)\(..\).*`\1/\2/`' | tr -d '\n'; echo $f; done;
./f > t
mkdir ../pacer2
./t | sed 's`\(..\)/\(..\)/\(.*\)`mkdir pacer2/\1\nmkdir pacer2/\1/\2\nmv pacer/\3 pacer2/\1/\2/\3`' > dirs

Published at DZone with permission of {{ articles[0].authors[0].realName }}, DZone MVB. (source)

Opinions expressed by DZone contributors are their own.

{{ tag }}, {{tag}},

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}
{{ parent.authors[0].realName || parent.author}}

{{ parent.authors[0].tagline || parent.tagline }}

{{ parent.views }} ViewsClicks