Over a million developers have joined DZone.

Moving files and folders into hashed subfolders

· Performance Zone

Download Forrester’s “Vendor Landscape, Application Performance Management” report that examines the evolving role of APM as a key driver of customer satisfaction and business success, brought to you in partnership with BMC.

The following will move a series of files into subfolders. It hashes the file names, building a two character, two folder deep hierarchy to split the files, e.g. a1/b2. The motivation for this is to split 500,000 folders into a manageable hierarchy, to avoid file system limits – in NTFS this is quite slow, and well over the 64,000 limit in ext4.

vi f
for f in `ls pacer/`; do echo -n $f | md5sum | sed 's`\(..\)\(..\).*`\1/\2/`' | tr -d '\n'; echo $f; done;
./f > t
mkdir ../pacer2
./t | sed 's`\(..\)/\(..\)/\(.*\)`mkdir pacer2/\1\nmkdir pacer2/\1/\2\nmv pacer/\3 pacer2/\1/\2/\3`' > dirs

See Forrester’s Report, “Vendor Landscape, Application Performance Management” to identify the right vendor to help IT deliver better service at a lower cost, brought to you in partnership with BMC.


Published at DZone with permission of Gary Sieling, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}