How to prepare S3 Cloud server on Ubuntu 16.04 and OEL7.X
What is Docker?
docker 基本教學 – 從無到有 Docker-Beginners-Guide
https://github.com/twtrubiks/docker-tutorial
Install Docker environment on Ubuntu 16.04
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add –
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add –
sudo apt-get install c
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add –
sudo add-apt-repository “deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable”
sudo apt-get update
apt-cache policy docker-ce
sudo apt-get install -y docker-ce
sudo systemctl status docker
sudo usermod -aG docker phillips
sudo docker search scality
sudo docker pull scality/s3server
sudo docker run -d –name s3server -p 8000:8000 scality/s3server
sudo docker stop s3server
sudo docker stats
sudo docker start s3server
sudo docker ps # help list container id
sudo docker exec -i -t $container_id bash # login to running image
apt-get update
apt-get install vim
vim package.json
“multiple_backend_test”: “CI=true S3BACKEND=file S3DATA=multiple mocha -t 20000 –recursive tests/multipleBackend”,
Install Docker environment on CentOS7/OEL7
#wget http://mirror.centos.org/centos/7/extras/x86_64/Packages/container-selinux-2.68-1.el7.noarch.rpm
#yum install selinux-policy-base selinux-policy-targeted
#rpm -ihv container-selinux-2.68-1.el7.noarch.rpm
sudo yum-config-manager –add-repo https://download.docker.com/linux/centos/docker-ce.repo
#yum install docker-ce
#docker images
#docker run -d –name s3server -p 8000:8000 scality/s3server
Login to s3Server docker system
sudo docker ps # finger out which one is s3server
sudo docker exec -i -t f9168e7bb704 bash
Show Log with s3Server running docker image
docker logs -f –tail 100 s3server
How to check Docker image location
Run df command in docker OS
root@a38cc05c505f:/# df | grep docker
/dev/mapper/docker-249:0-30860-7ef93373c452af140193f9e036dcca447512b21648d36454140bcb38b00c4349 26201600 374216 25827384 2% /
Check on Linux
[root@FS3147 devicemapper]# df -kh| grep \ 7ef93373c452af140193f9e036dcca447512b21648d36454140bcb38b00c4349
/var/lib/docker/devicemapper/mnt/7ef93373c452af140193f9e036dcca447512b21648d36454140bcb38b00c4349
[root@FS3147 devicemapper]# ls /var/lib/docker/devicemapper/mnt/7ef93373c452af140193f9e036dcca447512b21648d36454140bcb38b00c4349/rootfs/
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
Reference https://docs.docker.com/storage/storagedriver/#images-and-layers
How to use another storage for S3server storage input data
https://docs.docker.com/storage/bind-mounts/#start-a-container-with-a-bind-mount
We can prepare another large storage for docker image storage data
$sevice docker stop
$rsync -av –progress /var/lib/docker/volume/ /mnt/largesapce/
$mount –bind /mnt/largespace /var/lib/docker/volume
add new line to fstab
/mnt/largespace /var/lib/docker/volume none bind
How to install S3server for OEL7 # not for QA test
Install nodejs environment on OEL7.X
https://yum.oracle.com/oracle-linux-nodejs.html
Enabling Required Repositories
The installation instructions below require different Oracle Linux yum server repositories to be enabled, depending on what you are installing. For example, to enable the ol7_developer_nodejs6 repository:
$ sudo yum install -y yum-utils
$ sudo yum-config-manager –enable ol7_developer_nodejs6
If your yum configuration doesn’t have the required repository, grab the latest Oracle Linux yum configuration by running the these commands as the root user:
On Oracle Linux 7
# cd /etc/yum.repos.d
# mv public-yum-ol7.repo public-yum-ol7.repo.bak
# wget http://yum.oracle.com/public-yum-ol7.repo
On Oracle Linux 6
# cd /etc/yum.repos.d
# mv public-yum-ol6.repo public-yum-ol6.repo.bak
# wget http://yum.oracle.com/public-yum-ol6.repo
Installing Node.js
To install Node.js from this repository, ensure you have the latest Oracle Linux Yum server repo file and enable the appropriate repository.
To install Node.js:
$ sudo yum install nodejs
Define S3server storage location
Refer link https://github.com/scality/cloudserver#run-it-with-a-file-backend
By default the metadata files will be saved in the localMetadata directory and the data files will be saved in the localData directory within the ./S3 directory on your machine. These directories have been pre-created within the repository. If you would like to save the data or metadata in different locations of your choice, you must specify them with absolute paths. So, when starting the server:
mkdir -m 700 $(pwd)/myFavoriteDataPath
mkdir -m 700 $(pwd)/myFavoriteMetadataPath
export S3DATAPATH=”$(pwd)/myFavoriteDataPath”
export S3METADATAPATH=”$(pwd)/myFavoriteMetadataPath”
Run S3server with an in-memory backend
npm run mem_backend
PS: his is ideal for testing – no data will remain after container is shutdown.
[root@FS3147 S3]# npm run mem_backend
> @zenko/cloudserver@8.1.0-beta mem_backend /root/git/Scality/S3
> S3BACKEND=mem node index.js
{“name”:”S3″,”time”:1540267219174,”level”:”warn”,”message”:”scality kms unavailable. Using file kms backend unless mem specified.”,”hostname”:”FS3147″,”pid”:10978}
{“name”:”S3″,”time”:1540267219204,”bootstrap”:[“localhost”],”https”:false,”level”:”info”,”message”:”bucketclient configuration”,”hostname”:”FS3147″,”pid”:10978}
{“name”:”S3″,”time”:1540267219308,”https”:false,”level”:”info”,”message”:”Http server configuration”,”hostname”:”FS3147″,”pid”:10978}
{“name”:”S3″,”time”:1540267219311,”address”:”::”,”port”:8000,”pid”:10978,”level”:”info”,”message”:”server started”,”hostname”:”FS3147″}
{“name”:”S3″,”time”:1540267224311,”req_id”:”542591d8bc4086133efa”,”level”:”info”,”message”:”remote management disabled”,”hostname”:”FS3147″,”pid”:10978}
Reference https://github.com/scality/cloudserver/blob/master/docs/DOCKER.rst
S3BACKEND=file
When storing file data, for it to be persistent you must mount docker volumes for both data and metadata. See this section
S3BACKEND=mem
This is ideal for testing – no data will remain after container is shutdown.
We can edit the S3/package.json file to define S3BACKEND parameter
Support IPv6
Scaly S3server support IPv6 coding
https://github.com/scality/cloudserver/pull/482
Docker support IPv6
We have found the workaround method for setup scality s3server for IPv6 dual-stack
edit the S3server file
vim S3/config.json #add new entry for S3 server IPv6 name mapping
{
“port”: 8000,
“listenOn”: [],
“replicationGroupId”: “RG001”,
“restEndpoints”: {
“localhost”: “us-east-1”,
“127.0.0.1”: “us-east-1”,
“cloudserver-front”: “us-east-1”,
“s3.docker.test”: “us-east-1”,
“127.0.0.2”: “us-east-1”,
“aws.s3server.local”: “us-east-1”,
“2001:db8:100::199”: “us-east-1”,
“s3.amazonaws.com”: “us-east-1”,
“zenko-cloudserver-replicator”: “us-east-1”,
“lb”: “us-east-1”
},
….
edit /etc/hosts at S3server and OBD/FBD side # add IPv6 name resolve entry
2001:db8:100::199 aws.s3server.local s3.amazonaws.com
Run iscon object-storage-add via IPv6
iscon object-storage-add -s 127.0.0.1 -Ot AWS_S3 -Os verySecretKey1 -Oi accessKey1 -Ou http://aws.s3server.local:8000 -Ob vtl-bucket
Command: object-storage-add executed successfully.
For double check with aws-cli
aws –endpoint-url “http://aws.s3server.local:8000/” s3api create-bucket –bucket guumaster6
{
“Location”: “/guumaster6”
}
The issue is iscon cannot send out the IPv6 address format
$iscon object-storage-add -s 127.0.0.1 -Ot AWS_S3 -Os verySecretKey1 -Oi accessKey1 -Ou http://2001:db8:100::199:8000 -Ob vtl-bucket
AWS cli
https://docs.aws.amazon.com/en_us/cli/latest/userguide/installing.html
cat ~/.aws/config
[default]
access_key = accessKey1
secret_key = verySecretKey1
signature_v2 = True
user_https = False
cat ~/.aws/credentials
[default]
aws_access_key_id = accessKey1
aws_secret_access_key = verySecretKey1
Run aws cli to create bucket
#aws –endpoint-url http://192.168.1.29:8000/ s3api create-bucket –bucket guumaster
Check the S3server with tool called duplicity
Test Tool
Reference https://www.zenko.io/blog/backup-files-s3-server-duplicity/
http://duplicity.nongnu.org/index.html
https://github.com/librsync/librsync
Install librsync
git clone https://github.com/librsync/librsync.git
cd librsync/
cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Release .
sudo make install
sudo vi /etc/ld.so.conf.d/x86_64-linux-gnu.conf
Add “/usr/local/lib/x86_64-linux-gnu”
sudo ldconfig
Install duplicity
cd duplicity-0.7.18.2
sudo python setup.py install
sudo pip install boto
sudo vi /etc/boto.cfg
[Credentials]
aws_access_key_id = accessKey1
aws_secret_access_key = verySecretKey1
[Boto]
# If using SSL, set to True
is_secure = False
# If using SSL, unmute and provide absolute path to local CA certificate
# ca_certificates_file = /absolute/path/to/ca.crt
Run Test
duplicity full tmp/ “s3://172.22.17.29:8000/testbucket/”
Restore file Test
duplicity –file-to-restore NC.jar “s3://172.22.2.38:8000/testbucket/” tmp/NC.jar
Restore all Test
duplicity restore “s3://172.22.2.38:8000/testbucket/” tmp/
VTL/FBC iscon object-storage-add
Setup with VTL/FBD
iscon object-storage-add -s 127.0.0.1 -Ot AWS_S3 -Os verySecretKey1 -Oi accessKey1 -Ou http://172.22.17.29:8000 -Ob vtl-bucket
Tool for easy management Docker contanter(s)
Management Docker contanter