GlusterFS: host, is not befriended at the moment
Posted in Operating Systems, Linux, Fedora, Debian, Suse, Red Hat, Tools, Tips on Tuesday, June 4, 2013 by cam
When attempting to expand the replication distrubted file system of our cluster nodes I struck the error message:
[root@index03 ~]# gluster volume replace-brick drvol01 index04:/srv/gluster/drbrk02 index05:/srv/gluster/drbrk01 start index05, is not befriended at the moment
Which was strange considering the previous ‘peer probe’ from another host to join the new server (index05) was successfully:
[root@index04 ~]# gluster peer probe index05 Probe successful
I think the only clue was (on the last line):
[root@index03 drbrk02]# gluster peer status Number of Peers: 4 Hostname: index02 Uuid: 31c0f246-0a8b-4c68-8ffa-42b2e6bb42ce State: Peer in Cluster (Connected) Hostname: index04 Uuid: ca61b983-b8fd-4fcc-848a-6c33f102cd5c State: Peer in Cluster (Connected) Hostname: index01 Uuid: b9bc474c-0017-4191-82b9-5760ba0d00fc State: Peer in Cluster (Connected) Hostname: index05 Uuid: c6a7cb83-4c1f-42ff-8421-bea9cc911d0f State: Accepted peer request (Connected)
Anyway I confirmed that there were no firewall rules blocking the requests Doco: Gluster Firewall Ports and restarted the glusterd service on index05, which seems to have resolved the problem.
[root@index03 ~]# gluster peer status Number of Peers: 4 Hostname: index02 Uuid: 31c0f246-0a8b-4c68-8ffa-42b2e6bb42ce State: Peer in Cluster (Connected) Hostname: index04 Uuid: ca61b983-b8fd-4fcc-848a-6c33f102cd5c State: Peer in Cluster (Connected) Hostname: index01 Uuid: b9bc474c-0017-4191-82b9-5760ba0d00fc State: Peer in Cluster (Connected) Hostname: index05 Uuid: c6a7cb83-4c1f-42ff-8421-bea9cc911d0f State: Peer in Cluster (Connected)
In summary I believe there were firewall rules in place, when the ‘gluster peer probe index05′ was run. The rules were removed, but the gluster daemon on index05 then required a restart to become active in the cluster.