Note to self 003: BISystemUser Password Change

These days we faced an issue at a customer, and we were getting an error

[2014-09-17T05:55:58.000+02:00] [OracleBIServerComponent] [ERROR:1] [] [] [ecid: 00iOQuuYzVhB_6kLSyO5yc0001Ro000000,0:36357:6] [tid: 16d0] oracle.bi.security.service.SecurityServiceException: SecurityService::validateSystemUserCredentialsThe system user could not be authenticated.
[2014-09-17T05:55:58.000+02:00] [OracleBIServerComponent] [ERROR:1] [] [] [ecid: 00iOQuuYzVhB_6kLSyO5yc0001Ro000000,0:36357:6] [tid: 16d0] [nQSError: 43126] Authentication failed: invalid user/password.

And the users were not able to log into the OBIEE front end. All this points to an issue with the BISystemUser (default setup) not being correctly setup.

   

At first the customer tried (unsuccesfully):

1. Go to WLS Console (<http://<servername>:7001/console), Security Realms, myrealm, Users and Groups, Users. Locate the user BISystemUser, and change the password for this user
2. Log on to Enterprise Manager (http://<servername>:7001/em), expand Weblogic Domain, right- click bifoundation_domain, select Security, Credentials.
3. Under the "oracle.bi.system" folder, you will find the system.user credential. Edit this key, and type in the new password you set for the BISystemUser in step 1.
4. After making this change, things still did not work, even after restarting AdminServer, bi_server1 and the OPMN components.

The key to solving this issue was to carry out the steps above, with an important difference. We were seeing entries in the bi_server1.log file (in <MW_HOME>/user_projects/domains/bifoundation_domain/servers/bi_server1/logs) that the BISystemUser was being locked, so when changing the password for the user in the WLS Console, the BI components were still trying to log in with the "old" password, and therefore locking out the user. This is the error message:

####<Sep 17, 2014 9:49:08 AM CEST> <Notice> <Security> <CMA1CS0327> <bi_server1> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <75b304a5bd3e33a3:-7a4ce57c:1486b42f781:-8000-000000000004ada2> <1410940148242> <BEA-090078> <User bisystemuser in security realm myrealm has had 5 invalid login attempts, locking account for 30 minutes.>

   

These are the correct steps

1. Shut down all OPMN components (<MW_HOME>/instances/instancen/bin/opmnctl stopall)
2. Go to WLS Console (<http://<servername>:7001/console), Security Realms, myrealm, Users and Groups, Users. Locate the user BISystemUser, and change the password for this user
3. Log on to Enterprise Manager (http://<servername>:7001/em), expand Weblogic Domain, right- click bifoundation_domain, select Security, Credentials.
4. Under the "oracle.bi.system" folder, you will find the system.user credential. Edit this key, and type in the new password you set for the BISystemUser in step 2.
5. Stop the bi_server1 managed server and the WLS AdminServer
6. Restart the whole stack as usual.

Now the next question that pops up is why did it all go wrong in the first place - why did we need to change the password at all? This remains to be seen.....