My current version of fresh-hell is configuring mongod over SSL/TLS. I'm also utilizing this effort to convert my current working tables from MMAP to WiredTiger and converting the mongod.conf files over to the "new" YAML format.
My custom data-API framework, predicated on AMQP using RabbitMQ, has been vastly improved over the last year. Among other improvements, I've allowed for the configuration of a second data API service to be configured and exposed as another framework resource. When you combine our in-house source-code deployment environments (of which there are six) with the combination of two hosts data-API sourced VPC's, the SSL/TLS configuration combinations quickly climb into double-digits. Hence: fresh hell.
The engine conversion from MMAP to WiredTiger went flawlessly and, so far, I am jaw-drop-impressed over the speed improvements. (I register every query with a timer event - my query times have dropped down a full decimal point in some instances!)
The SSL/TLS configuration was largely successful although, not being a devOps guru, it took me a few days to get my head completely wrapped around the process of securing my mongo connections.
I did run into some issues during the deployment which caused no small amounts of confusion. Specifically, when I was attempting to load the server-certificate password using the PEMKeyPassword option in the configuration file.
What I was seeing was that, when I started the demon, I would get a success message (mongod start/running, process xxxxxx) but there would be no mongod process running. Additionally, even though I have my (development) logging set to the highest levels, not one error message appeared in the log to provide a hint as to why the service failed to start.
Instead of using the service command to start mongo, I started the service manually, invoking the application directly. I also asked for the highest level of verbosity while telling the mongod application to load my YAML configuration file:
$ mongod -vvv -f /etc/mongod.conf Error parsing YAML config file: yaml-cpp: error at line 40, column 1: illegal tab when looking for indentation try 'mongod --help' for more information
Which revealed why mongod was failing to start without posting an error message in the log file. Due to my rookie (non-existent) understanding of YAML syntax, combined with my vi editor not substituting spaces for tabs, I'd introduced a parsing error into the configuration file.
So, hey, the net-net of this article is to basically remind me of the following:
When mongod fails to start with no error output in the log files, start the service manually as a means of obtaining additional diagnostic information as to why the service failed to start.