Cloud-Agnostic Object Storage for Everybody!

during the agile design and architecture process of your new (web) service, two key questions will often be:

• how are we going to deal with object storage?

• is there a better way than getting locked into amazon s3 , digital ocean spaces, etc.?

yes, and yes! hello cloud-native world, meet minio , your private, on-your-own-terms object storage solution, on any public cloud or server.

this blog post will guide you on how to run your cloud object storage on any server with minio and cloud 66 backed by kubernetes .

what is object storage?

object storage, also known as object-based storage, is a computer data storage architecture that manages data as objects, as opposed to other storage architectures like file systems, which manage data as a file hierarchy, and block storage, which manages data as blocks within sectors and tracks.

each object typically includes the data itself, a variable amount of metadata, and a globally unique identifier.

what is minio?

minio is an object storage server, compatible with the amazon s3 cloud storage service. it is best suited for storing unstructured data such as photos, videos, log files, backups, and container/vm images. object size can range from a few kbs to a maximum of 5tb.

minio is like owning your own s3 cloud on your premises, under your data privacy laws, and on your own terms — and it's very easy to set up with cloud 66 on any public cloud or your own servers!

run minio locally for testing

before running minio in production, let's give it a test spin on your local box.

just install docker and docker-compose ( instructions here ) and use the following docker-compose.yml .

version: '3.1'  
services:  
 minio:
  image: minio/minio
  command: minio server /data 
  volumes:
   # mount to local disk
   - ./data:/data
  ports:
   # expose the ports
   - "9000:9000"
  environment:
   # setup the credentials
   minio_access_key: some_random_access_key
   minio_secret_key: very_secret_key

run docker-compose up and hit the end-point http://localhost:9000 , then use the minio_access_key and minio_secret_key credentials to login.

bam! you have your own s3-like object storage!

how to use it

minio is compatible with all s3 commands-if your service can talk to s3 you can talk to minio ! no code changes.

let's use an s3 command line tool called s3cmd as an example. you can find all the details here .

to play around with your local installment, edit the following fields in your s3cmd configuration file ~/.s3cfg

# setup endpoint
host_base = localhost:9000  
host_bucket = localhost:9000  
bucket_location = us-east-1  
use_https = false

# setup access keys
access_key =  some_random_access_key  
secret_key = very_secret_key

# enable s3 v4 signature apis
signature_v2 = falsefalse 

create a bucket in your brand new object storage:

add some files to your new bucket:

list the contents of your bucket:

or use the ui to manage your buckets and files.

run in production

it's time to run production on any cloud provider or on your own server.

we are going to deploy minio on our cloud 66 for containers offering — a complete solution for building, running, and maintaining containerized apps in production. this will allow you to run a multi-tenant setup on your own server(s).

• sign-up to cloud 66 , of course!
• create a new project and use the following service.yml

version: 2  
services:  
  minio:
    image: minio/minio:release.2017-08-05t00-00-53z
    command: minio server /data
    volumes:
      - /mnt/minio/data:/data
      - /mnt/minio/config:/root/.minio
    env_vars:
      minio_access_key: change_to_random_string
      minio_secret_key: change_to_random_string
    ports:
      - 9000:80:443

a service.yml is a cloud 66 service definition file that tells you which services we want to run on your infrastructure. ( for more about service.yml, check our documentation .)

given the ephemeral nature of containers, it's important to consider storage solutions to avoid data loss. that's the reason you find the volume statement in service.yml

 volumes:
      - /mnt/minio/data:/data
      - /mnt/minio/config:/root/.minio

choose to deploy in a production environment

choose which provider you want to deploy (make sure you entered your provider credentials) or bring your own server using our registered server feature .

hit deploy ! and wait until we're done with all the heavy lifting. after a couple of minutes, you will have deployed your own object storage, backed by kubernetes!

yes! kubernetes in space.

securing minio with ssl

of course, this installment is not secure because we are using http and not https. cloud 66 support let's encrypt (free!) ssl, or you can bring your own ssl.

again, we take care of all the heavy lifting and when the ssl certificate is created and installed, you have your minio running with ssl!

look! green lock!

test drive production

to start using your production minio service. edit the following fields in your s3cmd configuration file ~/.s3cfg

# setup endpoint
host_base = [your_domain_name]  
host_bucket = [your_domain_name]  
bucket_location = us-east-1  
# use https !!!
use_https = true

# setup access keys
access_key =  some_random_access_key  
secret_key = very_secret_key

# enable s3 v4 signature apis
signature_v2 = false  

check if you can create a bucket in your brand new object storage:

bam! object storage at your service.

summary

deploying your own s3-compatible object storage is as simple as buying a pair of shoes. it's fun running minio on any server! give us a spin a let us know what you think of our services.

start using the space!