Demistifying Certificate Generation using Ansible in Openshift
Background Ansible is the only available tools to manage deployment Openshift Origin Clusters, now known OKD. Some pitfalls were identified when the author tried to redeploy certificate in an Origin 3.6 cluster. Logging Issues We were trying to do some troubleshooting using the kibana-ops logging console, and were surprised to find it not working. Quick check on the logging project shows that many of the pods have warning status. Checking these pods (mostly elasticsearch instances) we found that the the health check script results in 'Elasticsearch node is not ready to accept HTTP requests yet' with http response code 000. The pods were at 90% storage, which I think caused by the curator unable to connect to the elastic instances (The curator is supposed to clean old index data files so the disk won't fill up). Further investigation results the curl command (adapted from this syntax from the chapter Aggregating Container Logs - Performing Administrative Elasticsearch Oper...