I needed to build a quick POC to show how to collect metrics from the Puppet server JVM via JMX. This is what I did, the problems I had and how I solved them.
1) Run Puppet server in a container.
That’s ok and it’s as simple as a
docker run command (assuming your machine is ready to run docker). But in order to enable JMX, you need to add some arguments.
In order to do that I decided to get the source code containing the Dockerfile to build the puppetserver-standalone image, baking in the necessary Java arguments.
See my changes to puppetlabs puppet-in-docker repo. I am certain that there are better ways to achieve the same without having to rebuild the image, directly changing the container from puppetlabs, but I don’t know how to do that.
2) build my own image:
cd puppet-in-docker/puppetserver-standalone docker build -t my-puppetserver-standalone .
3) run my image as a container:
docker run --rm -t --hostname container -p 1099:1099 my-puppetserver-standalone
4) make sure the host
container is resolve correctly.
As I use MacOS, that means adding the following line to my
5) run jconsole:
$ jconsole container:1099
6) click on “Insecure”.
This worked for me. Let me know if you get any problem.
0) a working Docker environment
1) In order to enable JMX you need to add the following arguments when starting the Java process:
-Djava.rmi.server.hostname=<hostname> -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=<port> -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.rmi.port=<port>
2) the machine running Jconsole must be able to “talk” to the port in
-Dcom.sun.management.jmxremote.port=<port> (see above) on the server (or container) running Puppet server (basically make sure there is no firewall blocking that port. This may not be relevant in this example, but I think it’s always worth mentioning.
3) name resolution for the RMI
<host> (based on my limited understanding[*]):
Jconsole will connect to JMX on the port under
RMI will then reply, normally with a randomly generated port number, unless you have specified the argument
-Dcom.sun.management.jmxremote.rmi.port=<port>, which pins the port (this helps when there is a firewall between Jconsole and JMX).
RMI will also provides a hostname (I don’t know what the default behaviour is, but the hostname too can be pinned with
-Djava.rmi.server.hostname=<hostname>). Make sure that the machine running Jconsole can resolve
<hostname>correctly, either locally of via DNS.
4) security concerns have been completely disregarded, but should not in real life systems (by adding support for SSL and authentication).
[*] ICBW (I could be wrong): please do get in touch if what I am writing is not true or wrong. I have done my best to be accurate and to report what I actually did, but there could always be something I forgot or got wrong.