Site Tools

New release available: 2022-07-31 "Igor". upgrade now! [52.2] (what's this?)

Installation and configuration of a protocol node under Spectrum Scale 4.2

Installation of ganesha NFS and SMB packages from the protcol node software suite. SMB packages must be installed as well otherwise mmces will not start.

[root@gprot1 ~]# rpm -qa | egrep "gan|smb"
[root@gprot1 ~]# 

set cesSharedRoot

set cesSharedRoot in the global config. cesSharedRoot can be located on an existing GPFS filesystem but it's recommended to place it in an own small GPFS filesystem for security reasons. This filesystem needs only to be mounted on the protoocol node.

mmcrfs cesroot -F /repo/config/stanza/titan2_cesroot.stanza -A yes \
       -B 64k -j scatter -m 1 -M 1 -R 1 -r 1 -n 128 -D posix -n 2 -T /mnt/cesroot

set CES root

Set the CES root in the global config. It can be located inside an existing filesystem, but for security reasons it should be placed into a separated filesystem. The min size of the cesSharedRoot is 4GB.

mmchconfig cesSharedRoot=/mnt/cesroot

enable CCR

enable ccr in the cluster; mandatory to configure CES

mmchcluster --ccr-enable

enable CES

set the protocol node to be ces enabled. The node must have the CES shared root mounted.

mmchnode --ces-enable -N 10

start up CES NFS

start up CES NFS service

mmces service enable nfs
mmces service start nfs

floating IP

set shared IP to be used as exporter interface for NFS and/or SMB. The IP must be resolvable (at least on the CES nodes and on the node where the command is issued)

mmces address add --ces-node gprot1 --ces-ip

check status

check CES status

[root@gpfs1 ~]# mmlscluster --ces

GPFS cluster information
GPFS cluster name:
GPFS cluster id:           1768562925710693003

Cluster Export Services global parameters
Shared root directory:                /mnt/cesroot
Enabled Services:                     NFS
Log level:                            3
Address distribution policy:          even-coverage

Node  Daemon node name            IP address       CES IP address list
10   gprot1            

[root@gpfs1 ~]# 

create user authentication

create user authentication using mmuserauth. This is the basic and mandatory setting. connections to LDAP or NIS or AD to be defined here.

mmuserauth service create --type userdefined --data-access-method file

check user authentication

[root@rwthpn2 ~]# mmuserauth service check -N cesNodes --rectify
File configuration is USER-DEFINED
Object not configured
[root@rwthpn2 ~]# 

create NFS export

[root@rwthpn2 ~]# mmnfs export add /rwth/vol1/nfsroot/ -c ",Protocols=3:4)"  Redirecting to /bin/systemctl stop  nfs-ganesha.service  Redirecting to /bin/systemctl stop  nfs-ganesha.service  Redirecting to /bin/systemctl start  nfs-ganesha.service  Redirecting to /bin/systemctl start  nfs-ganesha.service
NFS Configuration successfully changed. NFS server restarted on all NFS nodes.
[root@rwthpn2 ~]# 

show NFS export status

[root@rwthpn2 ~]# mmnfs export list
Path               Delegations Clients     
/rwth/vol1/nfsroot none 
[root@rwthpn2 ~]#  

[root@rwthpn2 ~]# mmnfs export list --nfsdefs /rwth/vol1/nfsroot/
Path               Delegations Clients     Access_Type Protocols Transports Squash      Anonymous_uid Anonymous_gid SecType PrivilegedPort DefaultDelegations Manage_Gids NFS_Commit 
/rwth/vol1/nfsroot none RW          3,4       TCP        ROOT_SQUASH -2            -2            SYS     FALSE          none               FALSE       FALSE      

[root@rwthpn2 ~]# 
mmces.txt · Last modified: 2022/07/10 09:34 by