Over a million developers have joined DZone.

Single Node Ceph Cluster Sandbox

DZone 's Guide to

Single Node Ceph Cluster Sandbox

I was asked about making something work with the Ceph S3API. I was able to proceed in creating a single node version of what was supposed to be six nodes.

· Database Zone ·
Free Resource

Well, I had not heard of the Ceph project until recently. I was asked about making something work with the Ceph S3API. After looking at these few different sites, I was able to proceed in creating a single node version of what was supposed to be six nodes. I wanted to be sure to storeshare the code for this.

sudo -i
useradd -d /home/cephuser -m cephuser
passwd cephuser
echo "cephuser ALL = (root) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/cephuser
sed -i s'/Defaults requiretty/#Defaults requiretty'/g /etc/sudoers
chmod 0440 /etc/sudoers.d/cephuser
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

yum install -y ntp ntpdate ntp-doc
ntpdate 0.us.pool.ntp.org
hwclock --systohc
systemctl enable ntpd.service
systemctl start ntpd.service
su - cephuser

#create a volume in AWS GUI - attach to instance
sudo blkid
ls -l /dev/xvd*

After the preparation steps, I was into the actual installation of Ceph. I found a few packages were missing both things that I wanted and a few requirements.

#install CEPH
yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum -y install wget mlocate 
yum-config-manager --enable rhui-REGION-rhel-server-optional

wget http://mirror.centos.org/centos/7/extras/x86_64/Packages/python-itsdangerous-0.23-2.el7.noarch.rpm
wget http://mirror.centos.org/centos/7/extras/x86_64/Packages/python-werkzeug-0.9.1-2.el7.noarch.rpm
wget http://mirror.centos.org/centos/7/extras/x86_64/Packages/python-flask-0.10.1-4.el7.noarch.rpm
rpm -i python-itsdangerous python-werkzeug python-flask

Now, let's get down to business.

sudo rpm -Uhv http://download.ceph.com/rpm-jewel/el7/noarch/ceph-release-1-1.el7.noarch.rpm
sudo yum update -y && sudo yum install ceph-deploy -y
mkdir cluster
cd cluster/
ceph-deploy new hostname
ceph-deploy install ceph-admin hostname
ceph-deploy mon create-initial
#didnt see this work BUT it didnt seem to matter
ceph-deploy gatherkeys hostname
#adding ODS
ceph-deploy disk list hostname
ceph-deploy disk zap hostname:/dev/xvdf
ceph-deploy osd prepare hostname:/dev/xvdf
ceph-deploy osd activate hostname:/dev/xvdf1
ceph-deploy disk list hostname

Probably one of the most important parts was the editing of the ceph.conf as described here. As well as the fact that I missed the 1 in the above ceph-deploy osd activate command. You have to look closely. To use the S3 endpoint you have to set up the ceph gateway (Rados server).

ceph-deploy gatherkeys hostname
ceph-deploy install --rgw hostanme
ceph-deploy rgw create hostname
netstat -plnt|grep 7480

Once I had done that, I simply needed a new user to make a connection the S3 commandline.

sudo radosgw-admin user create --uid="testuser" --display-name="First User”

    "user_id": "testuser",
    "display_name": "First User",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [],
    "keys": [
            "user": "testuser",
            "access_key": "***",
            "secret_key": "*****"
    "swift_keys": [],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    "user_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    "temp_url_keys": []

I then tested the Python program suggested by the docs:

sudo yum -y install python-boto

vi s3test.py

import boto.s3.connection

access_key = 'OZAOAWU1M55TGOWXF8F5'
secret_key = 'ApZR412bWipTK7eKHB2yyiJq5VmCOhfFcQ1qifjv'
conn = boto.connect_s3(
        host='', port=7480,
        is_secure=False, calling_format=boto.s3.connection.OrdinaryCallingFormat(),

bucket = conn.create_bucket('my-new-bucket')
for bucket in conn.get_all_buckets():
    print "{name} {created}".format(

Then S3. Of course, you need to install the S3 command line

aws s3 ls s3://my-new-bucket/ --endpoint-url http://XXX.XX.XX.XXX:7480

As you can see most calls to the gateway are supported:

Image title

database ,ceph ,cluster ,sandbox ,api

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}