How to use User and Group Filter in LDAP

[A-little-bit-of-data]

My requirement is to connect Kyuubi's user and group authentication to LDAP. Currently, the following configuration can only be used to authenticate a single user

kyuubi.authentication=LDAP
kyuubi.authentication.ldap.baseDN=dc=org
kyuubi.authentication.ldap.domain=apache.org
kyuubi.authentication.ldap.binddn=uid=kyuubi,OU=Users,DC=apache,DC=org
kyuubi.authentication.ldap.bindpw=kyuubi123123
kyuubi.authentication.ldap.url=ldap://hostname.com:389/

But according to the official documentation, it is configured as follows:

# Group Membership
kyuubi.authentication.ldap.groupClassKey=posixGroup
kyuubi.authentication.ldap.groupMembershipKey=memberUid
kyuubi.authentication.ldap.groupDNPattern=CN=%s,dc=test,dc=com
kyuubi.authentication.ldap.groupFilter=*


# User Search
#kyuubi.authentication.ldap.userDNPattern=uid=%s,dc=test,dc=com
#kyuubi.authentication.ldap.userFilter=*

# Custom Search
#kyuubi.authentication.ldap.customLDAPQuery=(&(objectClass=posixGroup)(memberUid=%s))

No single user can log in, the log is as follows:

 2025-01-05 18:34:06 DEBUG LdapSearchFactory:52 - Connecting using principal cn=admin,dc=test,dc=com to ldap server: ldap://XXX:389

 2025-01-05 18:34:06 DEBUG LdapSearch:52 - Executing a query: '(uid=evmli)' with base DNs dc=test,dc=com

 2025-01-05 18:34:06 INFO  GroupMembershipKeyFilter:64 - Authenticating user 'evmli' using GroupMembershipKeyFilter)

 2025-01-05 18:34:06 DEBUG LdapSearch:52 - Executing a query: '(uid=evmli)' with base DNs dc=test,dc=com

 2025-01-05 18:34:06 DEBUG LdapSearch:52 - Executing a query: '(&(objectClass=posixGroup)(|(memberUid=uid=evmli,ou=evm,dc=test,dc=com)(memberUid=evmli)))' with base DNs dc=test,dc=com

 2025-01-05 18:34:06 DEBUG GroupMembershipKeyFilter:52 - User uid=evmli,ou=evm,dc=test,dc=com member of: cn=evm,ou=evm,dc=test,dc=com,cn=predictive,ou=predictive,dc=test,dc=com

 2025-01-05 18:34:06 INFO  GroupMembershipKeyFilter:64 - Authentication failed based on user membership

 2025-01-05 18:34:06 ERROR TSaslTransport:280 - SASL negotiation failure

 javax.security.sasl.SaslException: Error validating the login

 	at org.apache.kyuubi.service.authentication.PlainSASLServer.evaluateResponse(PlainSASLServer.scala:80) ~[kyuubi-common_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.shaded.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:507) ~[kyuubi-relocated-thrift-0.4.0.jar:0.4.0]

 	at org.apache.kyuubi.shaded.thrift.transport.TSaslTransport.open(TSaslTransport.java:250) [kyuubi-relocated-thrift-0.4.0.jar:0.4.0]

 	at org.apache.kyuubi.shaded.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:44) [kyuubi-relocated-thrift-0.4.0.jar:0.4.0]

 	at org.apache.kyuubi.shaded.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:199) [kyuubi-relocated-thrift-0.4.0.jar:0.4.0]

 	at org.apache.kyuubi.shaded.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:227) [kyuubi-relocated-thrift-0.4.0.jar:0.4.0]

 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_412]

 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_412]

 	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_412]

 Caused by: javax.security.sasl.AuthenticationException: Authentication failed: User not a member of specified list

The corresponding users have actually been found here, but why does the verification still fail?

When I released the following configuration, I couldn't even find the corresponding relationship

# User Search
kyuubi.authentication.ldap.userDNPattern=uid=%s,dc=test,dc=com
kyuubi.authentication.ldap.userFilter=*

# Custom Search
kyuubi.authentication.ldap.customLDAPQuery=(&(objectClass=posixGroup)(memberUid=%s))

2025-01-05 18:48:25 DEBUG LdapSearchFactory:52 - Connecting using principal cn=admin,dc=test,dc=com to ldap server: ldap://XXXX:389

 2025-01-05 18:48:25 DEBUG LdapSearch:52 - Executing a query: '(&(objectClass=posixGroup)(memberUid=%s))' with base DNs dc=test,dc=com

 2025-01-05 18:48:25 INFO  CustomQueryFilter:64 - Trying with generic user search in ldap:(&(objectClass=posixGroup)(memberUid=evmli))

 2025-01-05 18:48:25 DEBUG LdapSearch:52 - Executing a query: '(&(objectClass=posixGroup)(memberUid=evmli))' with base DNs dc=test,dc=com

 2025-01-05 18:48:25 INFO  CustomQueryFilter:64 - Authentication failed based on result set from custom LDAP query

 2025-01-05 18:48:25 ERROR TSaslTransport:280 - SASL negotiation failure

 javax.security.sasl.SaslException: Error validating the login

 	at org.apache.kyuubi.service.authentication.PlainSASLServer.evaluateResponse(PlainSASLServer.scala:80) ~[kyuubi-common_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.shaded.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:507) ~[kyuubi-relocated-thrift-0.4.0.jar:0.4.0]

 	at org.apache.kyuubi.shaded.thrift.transport.TSaslTransport.open(TSaslTransport.java:250) [kyuubi-relocated-thrift-0.4.0.jar:0.4.0]

 	at org.apache.kyuubi.shaded.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:44) [kyuubi-relocated-thrift-0.4.0.jar:0.4.0]

 	at org.apache.kyuubi.shaded.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:199) [kyuubi-relocated-thrift-0.4.0.jar:0.4.0]

 	at org.apache.kyuubi.shaded.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:227) [kyuubi-relocated-thrift-0.4.0.jar:0.4.0]

 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_412]

 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_412]

 	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_412]

 Caused by: javax.security.sasl.AuthenticationException: Authentication failed: LDAP query from property returned no data

How should I configure LDAP so that I can find a single user and the corresponding group relationship, and the password can match the password in LDAP? If a user corresponds to multiple groups, and I set the sharing level to the group level, how do I choose which engine to connect to?

Now change the configuration to

# Group Membership Configuration
kyuubi.authentication.ldap.groupClassKey=posixGroup
kyuubi.authentication.ldap.groupMembershipKey=memberUid
kyuubi.authentication.ldap.groupDNPattern=cn=%s,dc=test,dc=com
kyuubi.authentication.ldap.groupFilter=evm,predictive
# User Search
#kyuubi.authentication.ldap.userDNPattern=uid=%s,dc=dfg,dc=com
#kyuubi.authentication.ldap.userFilter=*

# Custom Query
#kyuubi.authentication.ldap.customLDAPQuery=(&(objectClass=posixGroup)(memberUid=%s))

The log is as follows:

2025-01-06 11:37:26 DEBUG TSaslTransport:240 - SERVER: Start message handled

 2025-01-06 11:37:26 DEBUG TSaslTransport:173 - SERVER: Received message with status COMPLETE and payload length 21

 2025-01-06 11:37:26 DEBUG LdapSearchFactory:52 - Connecting using principal cn=admin,dc=,dc=com to ldap server: ldap://XXXX:389

 2025-01-06 11:37:26 DEBUG LdapSearch:52 - Executing a query: '(uid=evmli)' with base DNs dc=,dc=com

 2025-01-06 11:37:26 INFO  GroupMembershipKeyFilter:64 - Authenticating user 'evmli' using GroupMembershipKeyFilter)

 2025-01-06 11:37:26 DEBUG LdapSearch:52 - Executing a query: '(uid=evmli)' with base DNs dc=,dc=com

 2025-01-06 11:37:26 DEBUG LdapSearch:52 - Executing a query: '(&(objectClass=posixGroup)(|(memberUid=uid=evmli,ou=evm,dc=,dc=com)(memberUid=evmli)))' with base DNs dc=,dc=com

 2025-01-06 11:37:26 DEBUG GroupMembershipKeyFilter:52 - User uid=evmli,ou=evm,dc=,dc=com member of: cn=evm,ou=evm,dc=,dc=com,cn=predictive,ou=predictive,dc=,dc=com

 2025-01-06 11:37:26 DEBUG GroupMembershipKeyFilter:52 - GroupMembershipKeyFilter passes: user 'evmli' is a member of 'cn=evm,ou=evm,dc=,dc=com' group

 2025-01-06 11:37:26 INFO  GroupMembershipKeyFilter:64 - Authentication succeeded based on group membership

 2025-01-06 11:37:26 DEBUG LdapSearch:52 - Executing a query: '(uid=evmli)' with base DNs dc=,dc=com

 2025-01-06 11:37:26 DEBUG LdapSearchFactory:52 - Connecting using principal uid=evmli,ou=evm,dc=,dc=com to ldap server: ldap://xxxx:389

 2025-01-06 11:37:26 DEBUG TSaslTransport:134 - SERVER: Writing message with status COMPLETE and payload length 0

 2025-01-06 11:37:26 DEBUG TSaslTransport:263 - SERVER: Main negotiation loop complete

 2025-01-06 11:37:26 DEBUG TSaslServerTransport:207 - transport map does contain key org.apache.kyuubi.shaded.thrift.transport.TSocket@2e8c6b09

 2025-01-06 11:37:26 DEBUG TSaslTransport:427 - SERVER: reading data length: 299

 2025-01-06 11:37:26 DEBUG KyuubiTBinaryFrontendService:52 - TOpenSessionReq(client_protocol:HIVE_CLI_SERVICE_PROTOCOL_V10, configuration:{kyuubi.client.version=1.9.1, set:hivevar:kyuubi.engine.type=SPARK_SQL, set:hivevar:user=evmli, kyuubi.client.ipAddress=172.20.255.214, set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000, use:database=default})

 2025-01-06 11:37:26 INFO  KyuubiTBinaryFrontendService:64 - Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V10

 2025-01-06 11:37:26 INFO  KyuubiSessionManager:64 - Opening session for evmli@172.20.255.214

 2025-01-06 11:37:26 INFO  OperationLog:64 - Creating operation log file /opt/kyuubi/work/server_operation_logs/e54a579d-33b4-4dba-b5fa-94405b90e30c/5855343b-61f0-41ec-8468-91db1e897b48

 2025-01-06 11:37:26 INFO  KyuubiSessionManager:64 - evmli's KyuubiSessionImpl with SessionHandle [e54a579d-33b4-4dba-b5fa-94405b90e30c] is opened, current opening sessions 1

 2025-01-06 11:37:26 INFO  LaunchEngine:64 - Processing evmli's query[5855343b-61f0-41ec-8468-91db1e897b48]: PENDING_STATE -> RUNNING_STATE, statement:

 LaunchEngine

 2025-01-06 11:37:26 DEBUG TSaslTransport:466 - writing data length: 370

 2025-01-06 11:37:26 INFO  CuratorFrameworkImpl:235 - Starting

 2025-01-06 11:37:26 DEBUG CuratorZookeeperClient:183 - Starting

 2025-01-06 11:37:26 DEBUG ConnectionState:107 - Starting

 2025-01-06 11:37:26 DEBUG ConnectionState:255 - reset

 2025-01-06 11:37:26 INFO  ZooKeeper:442 - Initiating client connection, connectString=xxxx:2181 sessionTimeout=60000 watcher=org.apache.kyuubi.shaded.curator.ConnectionState@79695a29

 2025-01-06 11:37:26 DEBUG SaslServerPrincipal:74 - Canonicalized address to xxxx

 2025-01-06 11:37:26 INFO  ClientCnxn:1025 - Opening socket connection to server xxxx/xxxx:2181. Will not attempt to authenticate using SASL (unknown error)

 2025-01-06 11:37:26 INFO  ClientCnxn:879 - Socket connection established to xxxx/xxxx:2181, initiating session

 2025-01-06 11:37:26 DEBUG ClientCnxn:952 - Session establishment request sent on xxxx/xxxx:2181

 2025-01-06 11:37:26 INFO  ClientCnxn:1299 - Session establishment complete on server xxxx/xxxx:2181, sessionid = 0x100046a74831cf7, negotiated timeout = 40000

 2025-01-06 11:37:26 INFO  ConnectionStateManager:228 - State change: CONNECTED

 2025-01-06 11:37:26 DEBUG Groups:312 - GroupCacheLoader - load.

 2025-01-06 11:37:26 DEBUG TSaslTransport:427 - SERVER: reading data length: 100

 2025-01-06 11:37:26 DEBUG KyuubiTBinaryFrontendService:52 - TGetOperationStatusReq(operationHandle:TOperationHandle(operationId:THandleIdentifier(guid:58 55 34 3B 61 F0 41 EC 84 68 91 DB 1E 89 7B 48, secret:C2 EE 5B 97 3E A0 41 FC AC 16 9B D7 08 ED 8F 38), operationType:UNKNOWN, hasResultSet:false))

 2025-01-06 11:37:26 WARN  ShellBasedUnixGroupsMapping:210 - unable to return groups for user evmli

 org.apache.hadoop.security.ShellBasedUnixGroupsMapping$PartialGroupNameException: The user name 'evmli' is not found. id: ‘evmli’: no such user

 id: ‘evmli’: no such user

 

 	at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:294) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:207) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:97) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:51) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupList(Groups.java:387) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:321) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:270) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache.get(LocalCache.java:3962) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3985) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4946) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.security.Groups.getGroups(Groups.java:228) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.UserGroupInformation.getGroups(UserGroupInformation.java:1755) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1743) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.kyuubi.session.HadoopGroupProvider.groups(HadoopGroupProvider.scala:36) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.HadoopGroupProvider.primaryGroup(HadoopGroupProvider.scala:33) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.engine.EngineRef.<init>(EngineRef.scala:97) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.engine$lzycompute(KyuubiSessionImpl.scala:85) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.engine(KyuubiSessionImpl.scala:78) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.renewEngineCredentials(KyuubiSessionImpl.scala:269) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.engineCredentials$lzycompute(KyuubiSessionImpl.scala:76) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.engineCredentials(KyuubiSessionImpl.scala:76) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$2(KyuubiSessionImpl.scala:136) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$2$adapted(KyuubiSessionImpl.scala:133) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.ha.client.DiscoveryClientProvider$.withDiscoveryClient(DiscoveryClientProvider.scala:36) ~[kyuubi-ha_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1(KyuubiSessionImpl.scala:133) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSession.handleSessionException(KyuubiSession.scala:49) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.openEngineSession(KyuubiSessionImpl.scala:133) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.operation.LaunchEngine.$anonfun$runInternal$1(LaunchEngine.scala:60) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_412]

 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_412]

 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_412]

 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_412]

 	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_412]

 2025-01-06 11:37:26 DEBUG UserGroupInformation:1757 - Failed to get groups for user evmli

 java.io.IOException: No groups found for user evmli

 	at org.apache.hadoop.security.Groups.noGroupsForUser(Groups.java:200) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.Groups.access$400(Groups.java:75) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:334) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:270) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache.get(LocalCache.java:3962) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3985) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.thirdparty.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4946) ~[hadoop-client-runtime-3.3.6.jar:1.1.1]

 	at org.apache.hadoop.security.Groups.getGroups(Groups.java:228) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.UserGroupInformation.getGroups(UserGroupInformation.java:1755) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1743) ~[hadoop-client-api-3.3.6.jar:?]

 	at org.apache.kyuubi.session.HadoopGroupProvider.groups(HadoopGroupProvider.scala:36) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.HadoopGroupProvider.primaryGroup(HadoopGroupProvider.scala:33) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.engine.EngineRef.<init>(EngineRef.scala:97) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.engine$lzycompute(KyuubiSessionImpl.scala:85) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.engine(KyuubiSessionImpl.scala:78) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.renewEngineCredentials(KyuubiSessionImpl.scala:269) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.engineCredentials$lzycompute(KyuubiSessionImpl.scala:76) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.engineCredentials(KyuubiSessionImpl.scala:76) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$2(KyuubiSessionImpl.scala:136) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$2$adapted(KyuubiSessionImpl.scala:133) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.ha.client.DiscoveryClientProvider$.withDiscoveryClient(DiscoveryClientProvider.scala:36) ~[kyuubi-ha_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.$anonfun$openEngineSession$1(KyuubiSessionImpl.scala:133) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSession.handleSessionException(KyuubiSession.scala:49) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.session.KyuubiSessionImpl.openEngineSession(KyuubiSessionImpl.scala:133) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at org.apache.kyuubi.operation.LaunchEngine.$anonfun$runInternal$1(LaunchEngine.scala:60) ~[kyuubi-server_2.12-1.9.1.jar:1.9.1]

 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_412]

 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_412]

 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_412]

 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_412]

 	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_412]

 2025-01-06 11:37:26 WARN  HadoopGroupProvider:76 - There is no group for evmli, use the client user name as group directly

In fact, the corresponding information of the corresponding group is found in the log:

2025-01-06 11:37:26 DEBUG GroupMembershipKeyFilter:52 - User uid=evmli,ou=evm,dc=,dc=com member of: cn=evm,ou=evm,dc=,dc=com,cn=predictive,ou=predictive,dc=,dc=com

2025-01-06 11:37:26 DEBUG GroupMembershipKeyFilter:52 - GroupMembershipKeyFilter passes: user 'evmli' is a member of 'cn=evm,ou=evm,dc=,dc=com' group

Let's not talk about one user corresponding to multiple groups. Here, user 'evmli' is a member of 'cn=evm,ou=evm,dc=,dc=com' group

Why at the end 2025-01-06 11:37:26 WARN HadoopGroupProvider:76 - There is no group for evmli, use the client user name as group directly

Then I retreated from the group level to the user level and used evmli to start the spark SQL engine. However, I did not actually configure the spark configuration of evmli, only the evm configuration. If I want to implement such a function, does the existing kyuubi corresponding ldap configuration not support it?

[moelhoussein]

not sure if you got passed this, but your issue is that Spark Thrift is failing you not kyuubi. Kyuubi passes the auth, but your spark config is using the wrong group provider.

notice how your event says

org.apache.hadoop.security.ShellBasedUnixGroupsMapping$PartialGroupNameException

You need to set hadoop.security.group.mapping from the default org.apache.hadoop.security.ShellBasedUnixGroupsMapping to org.apache.hadoop.security.LdapGroupsMapping in either your core-site.xml or your spark confs.