Hadoop使用LDAP Group

延續著Zeppelin使用LDAP做認證這篇文章,當時在設定LDAP的時候有出一些小問題,那就是儘管我用LDAP帳號登入了某檯機器.在進行hadoop/hdfs等指令的時候帶的卻不是LDAP group而是supergroup

這樣會讓Apache Ranger等Hadoop ACL在進行group管理時無法正確捕捉到用戶的Group
後來查詢了一下HDP的官方說明
http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.0.0/bk_Ambari_Security_Guide/content/setting_up_hadoop_group_mappping_for_ldap_ad.html
必須在core-site.xml設定hadoop.security.group.mapping才行,不然預設hadoop捕捉的Unix上的user group
這裡改用LdapGroupsMapping來讓hadoop去抓LDAP的group,新增設定如下

core-site.xml
...
<property>
   <name>hadoop.security.group.mapping</name>
   <value>org.apache.hadoop.security.LdapGroupsMapping</value>
 </property>
 <property>
   <name>hadoop.security.group.mapping.ldap.base</name>
   <value>dc=example,dc=com</value>
 </property>
 <property>
   <name>hadoop.security.group.mapping.ldap.bind.password</name>
   <value>xxxx</value>
 </property>
 <property>
   <name>hadoop.security.group.mapping.ldap.bind.user</name>
   <value>cn=root,dc=example,dc=com</value>
 </property>
 <property>
   <name>hadoop.security.group.mapping.ldap.search.attr.group.name</name>
   <value>cn</value>
 </property>
 <property>
   <name>hadoop.security.group.mapping.ldap.search.attr.member</name>
   <value>member</value>
 </property>
 <property>
   <name>hadoop.security.group.mapping.ldap.search.filter.group</name>
   <value>(objectClass=groupOfNames)</value>
 </property>
 <property>
   <name>hadoop.security.group.mapping.ldap.search.filter.user</name>
   <value>(&amp;(objectClass=account)(uid={0}))</value>
 </property>
 <property>
   <name>hadoop.security.group.mapping.ldap.url</name>
   <value>ldap://ldap.example.com:389</value>
 </property>

LDAP的設定請參考Zeppelin使用LDAP做認證

上面core-site.xml設定好之後還得重開HDFS的service
之後用下面的指令刷新group mapping

$ hdfs dfsadmin -refreshUserToGroupsMappings
$ yarn rmadmin -refreshUserToGroupsMappings

之後用hdfs groups進行測試,若沒問題應該會看到group回來

$hdfs groups username

username:usergroup
comments powered by Disqus