Hi all,
While collecting accounts on LDAP, an error occurs as below yet all of them come when click the test button.
Could you please enlighten me what can be the reason?
Kind regards,
Tılsım
Hi all,
While collecting accounts on LDAP, an error occurs as below yet all of them come when click the test button.
Could you please enlighten me what can be the reason?
Kind regards,
Tılsım
Hi Regina,
Amount of data is <25, connection is OK. Collector failed a few seconds after starting. Here is the aveksaServer.log.
Could you please check it?
Thanks,
Tılsım
03/08/2018 14:34:27.166 INFO (Thread-8405 (HornetQ-client-global-threads-84946419)) [com.aveksa.server.xfw.ExecutionServiceProvider] Notified execution request for ExecutionContext[ID=38716 Type=AccountLoad Status=New NumTasks=4 tasks=ExecutionTask[TaskID=101663 RunID=38716 Source=1601 Type=DataObjectProcessing Status=New], ExecutionTask[TaskID=101662 RunID=38716 Source=1601 Type=DataPreProcessing Status=New], ExecutionTask[TaskID=101664 RunID=38716 Source=1601 Type=DataRelationshipProcessing Status=New], ExecutionTask[TaskID=101661 RunID=38716 Source=1601 Type=DataCollection Status=New]]
03/08/2018 14:34:27.174 INFO (Thread-8405 (HornetQ-client-global-threads-84946419)) [com.aveksa.server.xfw.ExecutionServiceProvider] Notified execution request for ExecutionContext[ID=38717 Type=EntitlementExplosion Status=New NumTasks=3 tasks=ExecutionTask[TaskID=101667 RunID=38717 Source=1601 Type=EntitlementExplosionPostProcessing Status=New], ExecutionTask[TaskID=101665 RunID=38717 Source=1601 Type=EntitlementExplosionPreProcessing Status=New], ExecutionTask[TaskID=101666 RunID=38717 Source=1601 Type=EntitlementExplosionProcessing Status=New]]
03/08/2018 14:34:27.218 INFO (Exec Task Consumer#1) [com.aveksa.server.xfw.TaskExecutor] Setting thread Thread[Exec Task Consumer#1,5,Execution Queue] on 575939022 method=Execute
03/08/2018 14:34:31.427 INFO (Thread-7757) [com.aveksa.client.component.communication.ChangeListHandler] Got new change item method=Run ChangeItem[ID=14393, type=CollectionRequest, source=1601, source-name=ACC_COL_LDAP_ORACLE_DIRECTORY]
03/08/2018 14:34:31.427 INFO (ApplyChangesRegularThread-18524) [com.aveksa.client.component.communication.ChangeListHandler] STARTING method=ApplyChanges subTask=Acting on a changeItem ChangeItem[ID=14393, type=CollectionRequest, source=1601, source-name=ACC_COL_LDAP_ORACLE_DIRECTORY]
03/08/2018 14:34:31.427 INFO (ApplyChangesRegularThread-18524) [com.aveksa.client.component.DefaultComponentManager] DCM132: Ignoring Event: com.aveksa.client.component.event.CollectionRequestEvent[source=com.aveksa.client.component.communication.DefaultCommunicationManager@3e15057f]
03/08/2018 14:34:32.792 ERROR (ApplyChangesRegularThread-18524) [com.aveksa.client.datacollector.framework.DataCollectorManager] DCM130: State of collector is: com.aveksa.client.datacollector.framework.DataCollectorManager$CollectorState@4c1483fa
So, it cannot be shut down.
03/08/2018 14:34:32.795 INFO (ApplyChangesRegularThread-18524) [com.aveksa.client.datacollector.framework.DataCollectorManager] DCM111: collector is com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector@1e3884df
03/08/2018 14:34:32.796 INFO (ApplyChangesRegularThread-18524) [com.aveksa.client.datacollector.framework.DataCollectorManager] DCM147: Booted Data Collector 1601
03/08/2018 14:34:32.815 INFO (ApplyChangesRegularThread-18524) [com.aveksa.collector.accountdata.LdapAccountDataReaderConfig] Context.REFERRAL : follow
03/08/2018 14:34:32.816 INFO (ApplyChangesRegularThread-18524) [com.aveksa.client.datacollector.framework.DataCollectorManagerFactory] DCMF64: Created DataCollectorManager for collector: 1601
03/08/2018 14:34:32.816 INFO (ApplyChangesRegularThread-18524) [com.aveksa.client.component.collector.DefaultCollectorManager] Created new data collector manager 1601
03/08/2018 14:34:32.817 INFO (ApplyChangesRegularThread-18524) [com.aveksa.client.datacollector.framework.DataCollectorManager] STARTING method=Collect CollectionMetaInfo[{ID=147, run_id=38716, collector_id=1601, test-run=false, collector_name=ACC_COL_LDAP_ORACLE_DIRECTORY}]
03/08/2018 14:34:32.848 INFO (ApplyChangesRegularThread-18524) [com.aveksa.collector.accountdata.LdapAccountDataReader] Fetching user data
03/08/2018 14:34:32.934 ERROR (ApplyChangesRegularThread-18524) [com.aveksa.collector.accountdata.LdapAccountDataReader] Error occurred in fetching users, Root Cause : [LDAP: error code 11 - Administrative Limit Exceeded]
javax.naming.LimitExceededException: [LDAP: error code 11 - Administrative Limit Exceeded]; remaining name 'dc=entp,dc=tgc'
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3172)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3033)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2840)
at com.sun.jndi.ldap.LdapNamingEnumeration.getNextBatch(LdapNamingEnumeration.java:147)
at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreImpl(LdapNamingEnumeration.java:216)
at com.sun.jndi.ldap.LdapNamingEnumeration.hasMore(LdapNamingEnumeration.java:189)
at com.aveksa.collector.accountdata.LdapAccountDataReader.getUsers(LdapAccountDataReader.java:674)
at com.aveksa.collector.accountdata.LdapAccountDataReader.access$100(LdapAccountDataReader.java:48)
at com.aveksa.collector.accountdata.LdapAccountDataReader$AccountDataDirectoryIterator.setContextAndGetUsers(LdapAccountDataReader.java:1247)
at com.aveksa.collector.accountdata.LdapAccountDataReader$AccountDataDirectoryIterator.<init>(LdapAccountDataReader.java:1164)
at com.aveksa.collector.accountdata.LdapAccountDataReader.createAccountDataDirectoryIterator(LdapAccountDataReader.java:230)
at com.aveksa.collector.accountdata.LdapAccountDataReader.getAccountIterator(LdapAccountDataReader.java:177)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collectData(AccountDataCollector.java:393)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collect(AccountDataCollector.java:302)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collect(AccountDataCollector.java:269)
at com.aveksa.client.datacollector.framework.DataCollectorManager.collect(DataCollectorManager.java:535)
at com.aveksa.client.component.collector.DefaultCollectorManager.actUpon(DefaultCollectorManager.java:203)
at com.aveksa.client.component.collector.DefaultCollectorManager.handle(DefaultCollectorManager.java:102)
at com.aveksa.client.component.event.DefaultEventManager.handle(DefaultEventManager.java:60)
at com.aveksa.client.datacollector.framework.SimpleEventSource.notifyListeners(SimpleEventSource.java:67)
at com.aveksa.client.component.communication.DefaultCommunicationManager.notifyEvent(DefaultCommunicationManager.java:377)
at com.aveksa.client.component.communication.ChangeListHandler.applyChanges(ChangeListHandler.java:364)
at com.aveksa.client.component.communication.ChangeListHandler.access$300(ChangeListHandler.java:58)
at com.aveksa.client.component.communication.ChangeListHandler$ChangeApplyingRunnable.run(ChangeListHandler.java:275)
at java.lang.Thread.run(Thread.java:745)
03/08/2018 14:34:32.934 INFO (ApplyChangesRegularThread-18524) [com.aveksa.collector.accountdata.LdapAccountDataReader] Data Read Exception happened :
com.aveksa.common.DataReadException: Error occurred in fetching users
at com.aveksa.collector.accountdata.LdapAccountDataReader.getUsers(LdapAccountDataReader.java:694)
at com.aveksa.collector.accountdata.LdapAccountDataReader.access$100(LdapAccountDataReader.java:48)
at com.aveksa.collector.accountdata.LdapAccountDataReader$AccountDataDirectoryIterator.setContextAndGetUsers(LdapAccountDataReader.java:1247)
at com.aveksa.collector.accountdata.LdapAccountDataReader$AccountDataDirectoryIterator.<init>(LdapAccountDataReader.java:1164)
at com.aveksa.collector.accountdata.LdapAccountDataReader.createAccountDataDirectoryIterator(LdapAccountDataReader.java:230)
at com.aveksa.collector.accountdata.LdapAccountDataReader.getAccountIterator(LdapAccountDataReader.java:177)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collectData(AccountDataCollector.java:393)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collect(AccountDataCollector.java:302)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collect(AccountDataCollector.java:269)
at com.aveksa.client.datacollector.framework.DataCollectorManager.collect(DataCollectorManager.java:535)
at com.aveksa.client.component.collector.DefaultCollectorManager.actUpon(DefaultCollectorManager.java:203)
at com.aveksa.client.component.collector.DefaultCollectorManager.handle(DefaultCollectorManager.java:102)
at com.aveksa.client.component.event.DefaultEventManager.handle(DefaultEventManager.java:60)
at com.aveksa.client.datacollector.framework.SimpleEventSource.notifyListeners(SimpleEventSource.java:67)
at com.aveksa.client.component.communication.DefaultCommunicationManager.notifyEvent(DefaultCommunicationManager.java:377)
at com.aveksa.client.component.communication.ChangeListHandler.applyChanges(ChangeListHandler.java:364)
at com.aveksa.client.component.communication.ChangeListHandler.access$300(ChangeListHandler.java:58)
at com.aveksa.client.component.communication.ChangeListHandler$ChangeApplyingRunnable.run(ChangeListHandler.java:275)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.naming.LimitExceededException: [LDAP: error code 11 - Administrative Limit Exceeded]; remaining name 'dc=entp,dc=tgc'
at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3172)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3033)
at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2840)
at com.sun.jndi.ldap.LdapNamingEnumeration.getNextBatch(LdapNamingEnumeration.java:147)
at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreImpl(LdapNamingEnumeration.java:216)
at com.sun.jndi.ldap.LdapNamingEnumeration.hasMore(LdapNamingEnumeration.java:189)
at com.aveksa.collector.accountdata.LdapAccountDataReader.getUsers(LdapAccountDataReader.java:674)
... 18 more
03/08/2018 14:34:32.935 ERROR (ApplyChangesRegularThread-18524) [com.aveksa.client.datacollector.framework.DataCollectorManager] FAILED method=Collect CollectionMetaInfo[{ID=147, run_id=38716, collector_id=1601, test-run=false, collector_name=ACC_COL_LDAP_ORACLE_DIRECTORY, data_file=/home/oracle/wildfly/standalone/tmp/vfs/deployment/deploymentf11b17c65020ecb8/aveksa.war-4f78fba92de474b2/WEB-INF/LocalAgent/collected_data/147.data}]
java.lang.RuntimeException
at com.aveksa.collector.accountdata.LdapAccountDataReader$AccountDataDirectoryIterator.setContextAndGetUsers(LdapAccountDataReader.java:1259)
at com.aveksa.collector.accountdata.LdapAccountDataReader$AccountDataDirectoryIterator.<init>(LdapAccountDataReader.java:1164)
at com.aveksa.collector.accountdata.LdapAccountDataReader.createAccountDataDirectoryIterator(LdapAccountDataReader.java:230)
at com.aveksa.collector.accountdata.LdapAccountDataReader.getAccountIterator(LdapAccountDataReader.java:177)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collectData(AccountDataCollector.java:393)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collect(AccountDataCollector.java:302)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collect(AccountDataCollector.java:269)
at com.aveksa.client.datacollector.framework.DataCollectorManager.collect(DataCollectorManager.java:535)
at com.aveksa.client.component.collector.DefaultCollectorManager.actUpon(DefaultCollectorManager.java:203)
at com.aveksa.client.component.collector.DefaultCollectorManager.handle(DefaultCollectorManager.java:102)
at com.aveksa.client.component.event.DefaultEventManager.handle(DefaultEventManager.java:60)
at com.aveksa.client.datacollector.framework.SimpleEventSource.notifyListeners(SimpleEventSource.java:67)
at com.aveksa.client.component.communication.DefaultCommunicationManager.notifyEvent(DefaultCommunicationManager.java:377)
at com.aveksa.client.component.communication.ChangeListHandler.applyChanges(ChangeListHandler.java:364)
at com.aveksa.client.component.communication.ChangeListHandler.access$300(ChangeListHandler.java:58)
at com.aveksa.client.component.communication.ChangeListHandler$ChangeApplyingRunnable.run(ChangeListHandler.java:275)
at java.lang.Thread.run(Thread.java:745)
03/08/2018 14:34:32.935 ERROR (ApplyChangesRegularThread-18524) [com.aveksa.client.datacollector.framework.DataCollectorManager] DCM281: Collection Failed: CollectionFailedEvent[cmi = CollectionMetaInfo[{ID=147, run_id=38716, collector_id=1601, test-run=false, collector_name=ACC_COL_LDAP_ORACLE_DIRECTORY, data_file=/home/oracle/wildfly/standalone/tmp/vfs/deployment/deploymentf11b17c65020ecb8/aveksa.war-4f78fba92de474b2/WEB-INF/LocalAgent/collected_data/147.data}] message = null cause = java.lang.RuntimeException]
java.lang.RuntimeException
at com.aveksa.collector.accountdata.LdapAccountDataReader$AccountDataDirectoryIterator.setContextAndGetUsers(LdapAccountDataReader.java:1259)
at com.aveksa.collector.accountdata.LdapAccountDataReader$AccountDataDirectoryIterator.<init>(LdapAccountDataReader.java:1164)
at com.aveksa.collector.accountdata.LdapAccountDataReader.createAccountDataDirectoryIterator(LdapAccountDataReader.java:230)
at com.aveksa.collector.accountdata.LdapAccountDataReader.getAccountIterator(LdapAccountDataReader.java:177)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collectData(AccountDataCollector.java:393)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collect(AccountDataCollector.java:302)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collect(AccountDataCollector.java:269)
at com.aveksa.client.datacollector.framework.DataCollectorManager.collect(DataCollectorManager.java:535)
at com.aveksa.client.component.collector.DefaultCollectorManager.actUpon(DefaultCollectorManager.java:203)
at com.aveksa.client.component.collector.DefaultCollectorManager.handle(DefaultCollectorManager.java:102)
at com.aveksa.client.component.event.DefaultEventManager.handle(DefaultEventManager.java:60)
at com.aveksa.client.datacollector.framework.SimpleEventSource.notifyListeners(SimpleEventSource.java:67)
at com.aveksa.client.component.communication.DefaultCommunicationManager.notifyEvent(DefaultCommunicationManager.java:377)
at com.aveksa.client.component.communication.ChangeListHandler.applyChanges(ChangeListHandler.java:364)
at com.aveksa.client.component.communication.ChangeListHandler.access$300(ChangeListHandler.java:58)
at com.aveksa.client.component.communication.ChangeListHandler$ChangeApplyingRunnable.run(ChangeListHandler.java:275)
at java.lang.Thread.run(Thread.java:745)
03/08/2018 14:34:32.936 ERROR (ApplyChangesRegularThread-18524) [com.aveksa.server.agent.message.ExceptionMessage]
java.lang.RuntimeException
at com.aveksa.collector.accountdata.LdapAccountDataReader$AccountDataDirectoryIterator.setContextAndGetUsers(LdapAccountDataReader.java:1259)
at com.aveksa.collector.accountdata.LdapAccountDataReader$AccountDataDirectoryIterator.<init>(LdapAccountDataReader.java:1164)
at com.aveksa.collector.accountdata.LdapAccountDataReader.createAccountDataDirectoryIterator(LdapAccountDataReader.java:230)
at com.aveksa.collector.accountdata.LdapAccountDataReader.getAccountIterator(LdapAccountDataReader.java:177)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collectData(AccountDataCollector.java:393)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collect(AccountDataCollector.java:302)
at com.aveksa.client.datacollector.collectors.accountdatacollectors.AccountDataCollector.collect(AccountDataCollector.java:269)
at com.aveksa.client.datacollector.framework.DataCollectorManager.collect(DataCollectorManager.java:535)
at com.aveksa.client.component.collector.DefaultCollectorManager.actUpon(DefaultCollectorManager.java:203)
at com.aveksa.client.component.collector.DefaultCollectorManager.handle(DefaultCollectorManager.java:102)
at com.aveksa.client.component.event.DefaultEventManager.handle(DefaultEventManager.java:60)
at com.aveksa.client.datacollector.framework.SimpleEventSource.notifyListeners(SimpleEventSource.java:67)
at com.aveksa.client.component.communication.DefaultCommunicationManager.notifyEvent(DefaultCommunicationManager.java:377)
at com.aveksa.client.component.communication.ChangeListHandler.applyChanges(ChangeListHandler.java:364)
at com.aveksa.client.component.communication.ChangeListHandler.access$300(ChangeListHandler.java:58)
at com.aveksa.client.component.communication.ChangeListHandler$ChangeApplyingRunnable.run(ChangeListHandler.java:275)
at java.lang.Thread.run(Thread.java:745)
03/08/2018 14:34:34.038 INFO (ApplyChangesRegularThread-18524) [com.aveksa.client.datacollector.framework.DataCollectorManager] SUCCESS method=Collect CollectionMetaInfo[{ID=147, run_id=38716, collector_id=1601, test-run=false, collector_name=ACC_COL_LDAP_ORACLE_DIRECTORY, data_file=/home/oracle/wildfly/standalone/tmp/vfs/deployment/deploymentf11b17c65020ecb8/aveksa.war-4f78fba92de474b2/WEB-INF/LocalAgent/collected_data/147.data}]
03/08/2018 14:34:34.038 INFO (ApplyChangesRegularThread-18524) [com.aveksa.client.component.fulfillment.AgentFulfillmentHandlerComponent] Handling event: com.aveksa.client.component.event.CollectionRequestEvent[source=com.aveksa.client.component.communication.DefaultCommunicationManager@3e15057f]
03/08/2018 14:34:34.039 INFO (ApplyChangesRegularThread-18524) [com.aveksa.client.component.fulfillment.AgentFulfillmentHandlerComponent] Ignoring event: com.aveksa.client.component.event.CollectionRequestEvent[source=com.aveksa.client.component.communication.DefaultCommunicationManager@3e15057f]
03/08/2018 14:34:34.754 INFO (Exec Task Consumer#1) [com.aveksa.server.xfw.TaskExecutor] UNSetting thread on 575939022 method=Execute
03/08/2018 14:34:34.755 INFO (Thread-18523) [com.aveksa.server.core.DBEventUtilities] Published 0 DB events for run 38716
03/08/2018 14:34:34.987 INFO (Thread-18523) [com.aveksa.server.xfw.ExecutionContext] Successful method=Execute 38716, AccountLoad
03/08/2018 14:34:35.042 INFO (Exec Task Consumer#0) [com.aveksa.server.xfw.TaskExecutor] Setting thread Thread[Exec Task Consumer#0,5,Execution Queue] on 1798779843 method=Execute
03/08/2018 14:34:35.206 INFO (Exec Task Consumer#0) [com.aveksa.server.xfw.TaskExecutor] UNSetting thread on 1798779843 method=Execute
03/08/2018 14:34:35.207 INFO (Exec Task Consumer#0) [com.aveksa.server.xfw.TaskExecutor] Setting thread Thread[Exec Task Consumer#0,5,Execution Queue] on 1892683201 method=Execute
03/08/2018 14:34:45.055 INFO (Exec Task Consumer#0) [com.aveksa.server.xfw.TaskExecutor] UNSetting thread on 1892683201 method=Execute
03/08/2018 14:34:45.057 INFO (Exec Task Consumer#0) [com.aveksa.server.xfw.TaskExecutor] Setting thread Thread[Exec Task Consumer#0,5,Execution Queue] on 649966949 method=Execute
03/08/2018 14:34:45.085 INFO (Exec Task Consumer#0) [com.aveksa.server.xfw.TaskExecutor] UNSetting thread on 649966949 method=Execute
03/08/2018 14:34:45.097 INFO (Thread-18526) [com.aveksa.server.core.DBEventUtilities] Published 0 DB events for run 38717
03/08/2018 14:34:45.109 INFO (Thread-18526) [com.aveksa.server.xfw.ExecutionContext] Successful method=Execute 38717, EntitlementExplosion
Looks like the root cause is:
LDAP: error code 11 - Administrative Limit Exceeded
I would check with the admins if there is some sort of a limit configured on returned search results.
Hi Boris,
Search returns in LDAP browser so I expect to see same result in RSA.
This could be an LDAP paging issue. Can you post your ldap filter you have configured?
Sometimes adding uid=* will fix this.
I agree, set page size in the collector to 100 and keep incrementing by 100 (max 1000) till you see failure.
Tried it 100 to 1000 as you said yet all attempts was failure.
Here is the filter. Also tried adding 'uid=*' and unfortunately result was same.
(&(objectclass=person)(employeetype=*)(employeenumber=*)(inetuserstatus=Active)(!(|(companycode=111)(companycode=222)(companycode=33)(companycode=44)(companycode=55)(companycode=77)(companycode=8888)(companycode=99)(companycode=123)(companycode=456)(companycode=789)(companycode=369))))
Additionally, when I try to add uid manually like (uid=xxx)(uid=yyy)(uid=zzz) all of them are collected but (uid=*) is not.
The problem is still remaining. Any help would be appreciated.
From you Linux server, try using ldapsearch (with the same filter) to see if you are getting the results or the error
ldapsearch -x -LLL -h [host] -D [user] -w [password] -b [base DN] -s sub "([filter])" [attribute list]After setting Lookthrough Limit to Unlimited, the error yields to [LDAP: error code 4 - Sizelimit Exceeded] and test of query returns 'Query reached limit of 2000 entries'. Is it still admin related? What do you suggest to get rid of this?
What LDAP are you using?
I suggest you adjust the settings for sizelimit as well.
The query reached limit of 2000 according to me just means you have put yor page filter to 2000 which is very high. Please lower that to 1000.
I would retest the collection after that.
If it still does not work then there are one last actions you can take.
Export your ldap to csv, set the charter code to UTF8
Check if a simple csv file collection also has problems.
IF so most likely you are running into characters which are causing problems during collection.
I would also strongly suggest to open up a support case.
We only pull a small number of records in the test to prove the general capability to reach the end point and be able to read from it. Running the account collector may run into issues due to data amount, connection issue to just name a few.
The aveksaServer.log should provide more insight, as well as the database summary logs for the run itself.