1. Create Ceph FS

Please refer UVS Manual v2.16.19 , from p106 Chapter for Ceph Filesystem for detail installation steps.

Down Link : https://ambedded.freshdesk.com/helpdesk/attachments/47091096336


Reference Demo Video : 

https://drive.google.com/file/d/1Kj70VALmKRP6rK-N-Z6hsF97ysZP1r27/view?usp=sharing


2. Create Subvolume correspond to Samba share folder


Ceph FS Name : cutfs

Share Folders :

1. SOT

2. IS

3. Broadcast


When Ceph Filesystem has been created , then we can create subvolumes correspond to the samba share folders . 


Steps :


-- Open a Terminal , ssh login to any MON nodes of ceph cluster 

-- Execute The following CLI command

Create a subvolume named [subvolume_name]  

$ ceph fs ls  => Get fs name [fsname]

$ ceph fs subvolumegroup ls [fsname] => Get subvolumegroup [subgroup]

$ ceph fs subvolume create  [fsname] [subvolume_name] --group_name [subgroup]  --mode 777

$ ceph fs subvolume ls cutfs cutfs.subgroup => Check if subvolume exists

i.e subvolume [fsname].subvolume is default created by UVS Create Cephfs


eg.

ceph fs ls  => Get fs name cutfs

ceph fs subvolumegroup ls cutfs => Get subvolumegroup cutfs.subgroup

ceph fs subvolume create  cutfs  cutfs.sot  --group_name cutfs.subgroup --mode 777

ceph fs subvolume create  cutfs  cutfs.is  --group_name cutfs.subgroup --mode 777

ceph fs subvolume create  cutfs  cutfs.broadcast  --group_name cutfs.subgroup --mode 777


Reference Demo Video : 
https://drive.google.com/file/d/1ATLBM5UIOmzqOifZ9MMXg-zJNdyYiSWf/view?usp=sharing


3. Mount Subvolume to Samba Gateway

Ceph FS Name : cutfs

Share Folders :

1. SOT (mount point /mnt/sot)

2. IS (mount point /mnt/is)

3. Broadcast (mount point /mnt/boradcast)


Steps : 


-- Open a Terminal , ssh login to any MON node

-- $ ceph auth get client.admin|grep key|awk '{print $3}' => Get admin keyring [keyadmin]

-- $ ceph mon dump -f json|jq -r ".mons[].addr"|awk -F '/' 'BEGIN {ORS=","} {print $1}' => Get mount MON String [monstring] (eliminate last character ",")

-- $ ceph fs ls  => Get fs name  [fsname] 

-- $ ceph fs subvolumegroup ls [fsname] => Get subvolumegroup [subgroup]

-- $ ceph fs subvolume getpath [fsname]  [subvolume] [subgroup] => Get subvolume path [subvolumepath]

-- Open another terminal , and ssh login to samba gateway

-- Insert the following string line to /etc/fstab 
    "  [monstring]:[subvolumepath]  [mount point] ceph name=admin,secret=[keyadmin],fs=[fsname],_netdev 0 2 "

-- $ mkdir -p [mount point]

-- $ mount -a

-- $ df -h => Check if subvolume has been mounted successfully


eg.

$ ceph auth get client.admin|grep key|awk '{print $3}' 

=> AQCUO/9iK/G/EhAAo9GtIQu3Wo176SDeTVFl9g==


$ ceph mon dump -f json|jq -r ".mons[].addr"|awk -F '/' 'BEGIN {ORS=","} {print $1}'

=> 10.133.3.16:6789,10.133.3.32:6789,10.133.3.8:6789,10.133.3.24:6789,10.133.3.40:6789 


$ ceph fs ls 

=> name: cutfs, metadata pool: metapool, data pools: [datapool ]


$ ceph fs subvolumegroup ls cutfs

=>  "name": "cutfs.subgroup"


$  ceph fs subvolume getpath cutfs cutfs.sot cutfs.subgroup

=> /volumes/cutfs.subgroup/cutfs.sot/6c85b367-d07f-4ad9-ae27-4fcc6112b7a0

Mount String :

10.133.3.16:6789,10.133.3.32:6789,10.133.3.8:6789,10.133.3.24:6789,10.133.3.40:6789:/volumes/cutfs.subgroup/cutfs.sot/6c85b367-d07f-4ad9-ae27-4fcc6112b7a0 


$ceph fs subvolume getpath cutfs cutfs.is cutfs.subgroup

=> /volumes/cutfs.subgroup/cutfs.is/6b07a83f-b723-45ae-82a9-5ec0a9105dd0

Mount String:

10.133.3.16:6789,10.133.3.32:6789,10.133.3.8:6789,10.133.3.24:6789,10.133.3.40:6789:/ /volumes/cutfs.subgroup/cutfs.is/6b07a83f-b723-45ae-82a9-5ec0a9105dd0  /mnt/is ceph name=admin,secret=AQCUO/9iK/G/EhAAo9GtIQu3Wo176SDeTVFl9g==,fs=cutfs,_netdev 0 2


$  ceph fs subvolume getpath cutfs cutfs.broadcast cutfs.subgroup

/volumes/cutfs.subgroup/cutfs.broadcast/5efcc1af-9d1a-4f81-896d-345897fae9d4

Mount String :

10.133.3.16:6789,10.133.3.32:6789,10.133.3.8:6789,10.133.3.24:6789,10.133.3.40:6789:/volumes/cutfs.subgroup/cutfs.broadcast/5efcc1af-9d1a-4f81-896d-345897fae9d4  /mnt/broadcast ceph name=admin,secret=AQCUO/9iK/G/EhAAo9GtIQu3Wo176SDeTVFl9g==,fs=cutfs,_netdev 0 2


$ echo "[Mount String]" >> /etc/fstab

$ mkdir -p /mnt/sot ; mkdir -p /mnt/is ; mkdir -p /mnt/broadcast

$ mount -a

$ df -h


Reference Demo Video : 
https://drive.google.com/file/d/1jbKn-Mq8KrleHmMCekQt-pMun_B535Y6/view?usp=sharing


4. Configure smb.conf to add Samba Share Folder

Share Folders :

1. SOT (mount point /mnt/sot)

2. IS (mount point /mnt/is)

3. Broadcast (mount point /mnt/boradcast)


Steps:


--  Open a Terminal , ssh login to Samba gateway

--  Open file /etc/samba/smb.conf with root permission via text editor (eg. vi , vim ,...)

--  Add the following samba configuration to /etc/samba/smb.conf 
    i.e. This will be an example , please change configuration according to your environment 

 

Given 

Share Folder name : $ShareName 

Mount Poiint : $MountPoint

 

[$ShareName]

acl_xattr:ignore system acl = Yes

acl allow execute always = Yes

acl group control = Yes

inherit acls = Yes

inherit owner = windows and unix

inherit permissions = Yes

path = $MountPoint

read only = No


eg.


[SOT]

acl_xattr:ignore system acl = Yes

acl allow execute always = Yes

acl group control = Yes

inherit acls = Yes

inherit owner = windows and unix

inherit permissions = Yes

path = /mnt/sot

read only = No


[IS]

acl_xattr:ignore system acl = Yes

acl allow execute always = Yes

acl group control = Yes

inherit acls = Yes

inherit owner = windows and unix

inherit permissions = Yes

path = /mnt/is

read only = No


[Broadcast]

acl_xattr:ignore system acl = Yes

acl allow execute always = Yes

acl group control = Yes

inherit acls = Yes

inherit owner = windows and unix

inherit permissions = Yes

path = /mnt/broadcast

read only = No


-- Save and Close text editor 

-- $ systemctl daemon-reload

-- $ systemctl restart nmbd smbd

-- Check connections in Windows System


Reference Demo Video : 
https://drive.google.com/file/d/18YmhMEdXde8UlKG71kgaNvBgnCnV42AA/view?usp=sharing


5. Set Quota to Samba Share Folder


Please refer the document 
`` Setup CEPH FS quota for shared folder(NFS/SAMBA based) ``

Link : https://ambedded.freshdesk.com/a/solutions/articles/47001171677


Once the cephfs and subvolume has been setup , we can only focus on step 1 & 4