rhc app commands are failing with "Resources Temporarily Unavailable" errors. What should I do?


When trying to run a rhc command against an app (note the -d for debug log level), the following errors may occur:  
> rhc app restart -a testapp -d
Password: *********
Contacting https://openshift.redhat.com
Problem reported from server. Response code was 500.
runuser: cannot set user id: Resource temporarily unavailable
Cartridge return code: 125
Exit Code: 1
api_c: placeholder
broker_c: namespacerhloginsshapp_uuiddebugaltercartridgecart_typeactionapp_nameapi
API version:    1.1.3
Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.

Connections via ssh may fail as well.


"Resources Temporarily Unavailable" is a sign that your application has ran out of threads/processes to handle requests such as rhc app commands, or ssh connections.  

You will need to force-stop your application, and debug your code to ensure that you're closing threads/processes properly:  
> rhc app force-stop -a {appName}

Afterwards, you can ssh onto your gear and review your resource utlization with this FAQ:  https://openshift.redhat.com/community/faq/how-many-applications-can-i-deploy-and-what-resource-limitations-would-they-have