Are you taking advantage of Secrets Cache with Password Safe for DR/Break-Glass Scenarios?

Yes, it works great for secret safe password but there are a couple of pit falls to avoid with password safe.  When the cache requests a password from the password safe, the request will use the “default” request time limit.  If that credential is configured to rotate on checkin, the credential will rotate and the password cache will have an old password.  This is a timing issue with no active requests triggering a password change and the cache making a new request.  You have to enable api override rotation and modify the default “rotationpolicy” in the password cache configuration.  The password safe cache will hold an active request for the credential, so when a user checks in a password, it will not rotate until the password safe checks in it’s request which is set to the default request time.  It will then rotate and the cache will checkout the password so quickly the cache will retain the old password for 1 round of the “default” request time.  This cache is really intended to have DR accounts that are not used daily.  

Hey Mike! Great points around the default request time limit and the credential rotating on check-in appreciate you providing insight.

Regarding the timing issue, Password Safe will wait for all requests (including the cache request) to expire before a queued post-release password rotation occurs. Assuming the rotation is timely (and not delayed due to many processing changes or other reasons), the cache should obtain the new password when it creates a new request for credentials on its next cycle (default=5 mins). This means you shouldn't need to enable API override rotation and modify the default “rotationpolicy” in the password cache configuration. If you’re getting different results, I’d suggest reaching out to our Support team to investigate further.

To your last point, you’re correct -- Password Safe Cache will hold an active request for the credential and not rotate that until the request is checked in (set to the default request time). This is the default/expected behavior for all queued/scheduled password changes, the system can’t submit the process to rotate the password until all requests have expired. And finally, you’re absolutely right about the DR accounts, these should not be used daily.

I did open a case with support and they pointed me to KB0016990.  The got you is PS cache checks in the request, which will then allow the password to rotate.  The password safe will then que a rotation.  The cache immediately checks out a new request/password, but the change que hasn’t processed the change.  This means the password safe cache has the old password.  That will remain until the request expires and renews.  Many people attempt to use the cache as a password safe backup which it’s not designed for.  If you leave accounts with a default request of 2 hours, the cache uses a 2 hour request.  If you have many accounts in your cache, you can reduce the noise in password safe by expending the default request, but you run into this little issue when credentials rotate.

Mike, this is the exact same behavior I've had in our instance of the cache. The timing issue causes the cache to be a rotation behind. 

Having implemented Password/Secrets Cache (PWC) in numerous client environments, it should be understood that this solution is a DR/Break Glass solution, not an offline password safe backup. The number of accounts sent to PWC should be very limited. Use only the accounts that are deemed business critical to restoring the environment. Think of the iDRAC, ESXi, and other business critical managed accounts. Also, PWC updates every 5 minutes (can be adjusted via registry) so take this into account in light of the “stale password” discussion above. Lastly, it is best practice to have multiple PWC servers so as to not have all your “DR Eggs” in one basket. 

Articles regarding Secrets Safe cache:

Password Safe Secrets Cache overview and Windows configuration KB0016990

Password Safe Secrets Cache for Windows - Step-by-step for configuring, testing and troubleshooting
KB0020586

Why does the Password Safe Secrets Cache user login every 5 minutes? KB0020682