Principal Software Engineer/Architect, motorcyclist, drummer, and artist
The easiest way that I have seen so far for doing so is to use tail and simply read everything except the first three bytes (start reading at the 4th byte), as follows:
tail --bytes=+4 text_file.txt text_file-wo-bom.txtI am working on a project and have decided to use Avro for the data serialization format.
I encountered the following error when trying to set up the unit test to test the mapper implementation through Eclipse:
java.lang.NoSuchMethodError: org.apache.avro.generic.GenericData.createDatumWriter(Lorg/apache/avro/Schema;)Lorg/apache/avro/io/DatumWriter;
at org.apache.avro.hadoop.io.AvroSerialization.getSerializer(AvroSerialization.java:114)
at org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:82)
at org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:67)
at org.apache.hadoop.mrunit.internal.io.Serialization.copy(Serialization.java:98)
at org.apache.hadoop.mrunit.internal.io.Serialization.copyWithConf(Serialization.java:111)
at org.apache.hadoop.mrunit.TestDriver.copy(TestDriver.java:676)
at org.apache.hadoop.mrunit.TestDriver.copyPair(TestDriver.java:680)
at org.apache.hadoop.mrunit.MapDriverBase.addInput(MapDriverBase.java:120)
at org.apache.hadoop.mrunit.MapDriverBase.addInput(MapDriverBase.java:130)
at org.apache.hadoop.mrunit.MapDriverBase.addAll(MapDriverBase.java:141)
at org.apache.hadoop.mrunit.MapDriverBase.withAll(MapDriverBase.java:247)
at com.ryanchapin.hadoop.mapreduce.mrunit.UserDataSortTest.testMapper(UserDataSortTest.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
The newer (I am currently using Mars, 4.5.0) versions of Eclipse provide very good tools for configuring the visibility of whitespace characters in code.
To customize your settings go to Window -> Preferences -> General -> Editors -> Text Editors.
On that page there will be checkbox option next to “Show whitespace characters (configure visibility).
Clicking on the ‘configure visibility’ link will allow you to choose what is shown and the opacity of the whitespace characters, which is a really → Continue reading “Configuring Hidden, Invisible, or Whitespace Characters in The Eclipse Text Editor”
I am putting together a chrooted installation of named and ran into a problem whereby attempting to generate an rndc.key with rndc-confgen just hangs, never returning and not generating a key.
After doing some searching I discovered that I needed to run the command as follows:
rndc-confgen -a -r /dev/urandom -t /var/named/chrootWhich outputs the following, generating the key file that I expected.
wrote key file "/etc/rndc.key"
wrote key file "/var/named/chroot/etc/rndc.key"→ Continue reading “[SOLVED] Configuring chrooted bind and rndc-confgen Hangs Not Generating a Key”
Sometime your java applilcation will fail and generate an enormous heap dump. One that may be too large to be able to transfer to your local machine and to analyze for lack of RAM, time or both.
One solution is to install the MAT tool on the remote server and generate an HTML output of the analysis to download and view locally. This saves the headache of attempting to get X Windows installed on the remote machine and get all → Continue reading “Using the Eclipse Memory Analyzer (MAT) Remotely on Large Heap Dumps”
Following is a quick-start for getting a Chef and Test Kitchen environment set up to start writing and testing Chef cookbooks. This is an work in progress, so please if you find something that is incorrect or does not work as expected, send me an e-mail.
Development Environment Set-up:
Unless you are a Ruby developer and already have a complete environmenet set-up the easiest thing to do is to install the Chef Development Kit, https://downloads.chef.io/chef-dk/.
After downloading and → Continue reading “Quick-Start and Minimal Cheat Sheet for Chef and Test Kitchen”
When executing ‘search-and-replace’ commands on ASCII under Linux, Unix (or *nix) operating systems, sed works or most cases and makes for reasonably straightforward reading of the script.
If you want to replace multiple lines of text with multiple lines of text, following is a perl one-liner that does the trick and is much easier to wrangle than trying to do it in sed.
perl -i -pe "BEGIN{undef $/;} s:${EXISTING_LINES}:${REPLACEMENT_TEXT}:smg" file.txtFollowing is *nix a command that you can use to add the contents of a source text file to the start of another text file (the source file).
echo -e '0r <source_file_name\nw' | ed -s <target_file_nameOften administrators would like to configure a Samba share that enables users to have the same access to any files within the share. Without some additional configuration, directories and files created by one user will not have the r/w permissions for other users that have access to that same share.
The end goal is to have a Samba share in which any new files are created with r/w permissions for a specific group to which all of the members of → Continue reading “Creating a Samba Share and Configuring an Access Control List for a Shared Directory Under Linux”
To update the versions of all of the poms in a multiple module project use the versions-maven plugin.
To update
mvn versions:set -DnewVersion=1.4.0-SNAPSHOTWill modify all of the versions of each of the poms to the version specified. It will create a pom.xml.versionsBackup for each pom file that it modified. You can then examine each to make sure that it is as you intended.
If you want, you can revert your change with
mvn versions:revertIf you are satisfied with → Continue reading “Updating all of the pom.xml Version Numbers in a Multi-Module Maven Project”