Author: rchapin

Solution for Executing Native Process from Java that Requires sudo

Posted on by rchapin

If you are building a Java program that requires the ability to execute native commands on the machine which require sudo it requires some additional considerations other than just writing the Java code.

The problem is that sudo, by default, requires a tty for executing sudo such that a password can entered.  Even if you configure sudoers to grant NOPASSWD access to a specific command you will still get the following error

sudo: sorry, you must have a tty to
→ Continue reading “Solution for Executing Native Process from Java that Requires sudo”

[SOLVED] Ambari There are no DataNodes to do rolling restarts when there are DataNodes that do need a restart

Posted on by rchapin

When maintaining a Hadoop cluster, you will need to restart various service from time-to-time when/if you update Hadoop configurations.

I ran into a problem today with Ambari where I wanted to do a rollling restart of all of my DataNodes, but when I clicked on the “Restart DataNodes” entry in the “Restart” drop down the dialog indicated “There are no DataNodes to do rolling restarts”.

This was clearly incorrect.

It did not take me too long to figure out that → Continue reading “[SOLVED] Ambari There are no DataNodes to do rolling restarts when there are DataNodes that do need a restart”

[SOLVED] Unable to Connect to ambari-metrics-collector Issues

Posted on by rchapin

I was having some issues with the ambari-metrics family of services on a ‘pseudo-distributed’ cluster that I have installed on my workstation.

The symptoms were:

1.  Ambari indicated the following CRITICAL errors in the Ambari Dashboard under the Ambari Metrics section

Connection failed: [Errno 111] Connection refused to rchapin-wrkstn:6188

2.  After attempting to restart the ambari-metrics-collector via either the Ambari Dashboard or through the commandline (# ambari-metrics-collector [stop|start]) you see the following (similar) messages in the ambari-metrics-collector.log

2016-09-02 12:15:37,505 INFO
→ Continue reading “[SOLVED] Unable to Connect to ambari-metrics-collector Issues”

Dynamically Instantiating Classes in Python

Posted on by rchapin

A common design pattern is to define a class or list of classeses in configuration such that at runtime the classes can be dynamically instantiated.

I’ve done this before in many other languages and had need to do so today in Python (2.7.11)

It seems as though the clean way to do so is by using the Pyton importlib module.  By using it, it enables you to cleanly dynamically import sub modules

Following is an example:

import importlib

klass_1_module =
→ Continue reading “Dynamically Instantiating Classes in Python”

Java PowerMock Could not reconfigure JMX java.lang.LinkageError Solution

Posted on by rchapin

If you are using Mockito and PowerMock to build mocks for your Java tests and you run into the following error:

2016-05-05 17:31:20,204 main ERROR Could not reconfigure JMX java.lang.LinkageError: loader constraint violation: loader (instance of org/powermock/core/classloader/MockClassLoader) previously initiated loading for a different type with name "javax/management/MBeanServer"
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
        at org.powermock.core.classloader.MockClassLoader.loadUnmockedClass(MockClassLoader.java:238)
        at org.powermock.core.classloader.MockClassLoader.loadModifiedClass(MockClassLoader.java:182)
        at org.powermock.core.classloader.DeferSupportingClassLoader.loadClass(DeferSupportingClassLoader.java:70)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

Simply add the following as a class level annotation:

@PowerMockIgnore( {"javax.management.*"} )

→ Continue reading “Java PowerMock Could not reconfigure JMX java.lang.LinkageError Solution”

Blacklisting Kernel Modules

Posted on by rchapin

Following is a walkthrough on how to blacklist a kernel module.  The specific example is blacklisting the nouveau driver so that I can install the OEM Nvidia driver.

1. First, blacklist the nouveau driver: Add a line to the textfile /etc/modprobe.d/nouveau-blacklist.conf that contains they keyword ‘blacklist’ and the name of the driver

blacklist nouveau

2. Rebuild the initramfs image file.  First, backup existing initramfs

mv initramfs-3.10.0-327.18.2.el7.x86_64.img initramfs-3.10.0-327.18.2.el7.x86_64.img_2016-06-09.bak

Build new initramfs

dracut -v /boot/initramfs-$(uname -r).img $(uname -r)

3. Reboot the system → Continue reading “Blacklisting Kernel Modules”

How to Return Hive Query Results Similary to MySQL \G in One Vertical Column

Posted on by rchapin

When trying to look at data in a database with really wide rows even just selecting 1 row to see the data is nearly impossible to understand when the single row wraps 7 or 8 times.

MySQL offers the ‘\G’ option to display the output in a single column.

The corresponding method in Hive is to execute the following set command:

!set outputformat vertical
SELECT something FROM some table;
→ Continue reading “How to Return Hive Query Results Similary to MySQL \G in One Vertical Column”

How to Configure a User Account in Active Directory So that the Password Never Expires

Posted on by rchapin

Using ADSI Edit, navigate to the user in question.  Right-click on the user and select Properties.

Then, scroll down to the ‘userAccountControl’ property and click the Edit button.

Enter 66048 into the field and then click ‘OK’ and then ‘Apply’.

After closing the edit window, scroll to the right to confirm that the value indicates the following

'0x10200 = (NORMAL_ACCOUNT|DONT_EXPIRE_PASSWORD)'
→ Continue reading “How to Configure a User Account in Active Directory So that the Password Never Expires”

Using netcat to Mock a RESTful Webservice that Returns JSON

Posted on by rchapin

Let’s say that you are working on a part of a project that needs to consume some JSON data from a forthcoming HTTP service.  That sub-system is being developed by one of your colleagues and is not yet ready for you to stand-up to develop against.

You can use netcat to mock the webservice and return some static JSON data for which you can develop and test against with a simple one-liner.

First, put together your JSON in a file, → Continue reading “Using netcat to Mock a RESTful Webservice that Returns JSON”

How to Use Credentials That Contain Special Characters with curl

Posted on by rchapin

In order to execute curl commands to endpoints with passwords that contain special characters, the cleanest way that I have found to do so is to Base64 encode the authentication string for curl and then pass an Authorization request header along with the request.

In this example the credentials are uid ‘rchapin’ and passwd ‘abc123!@#’.  Normally we would pass this to curl as follows:

$ curl -u rchapin:abc123!@# -X GET https://some-endpoint:443

However, this will not work and the password will → Continue reading “How to Use Credentials That Contain Special Characters with curl”

© 2005 Ryan Chapin