atom.xml

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">

  <title><![CDATA[Systems and Automation]]></title>
  <link href="http://blog.fretzel.net/atom.xml" rel="self"/>
  <link href="http://blog.fretzel.net/"/>
  <updated>2015-01-07T13:06:35-06:00</updated>
  <id>http://blog.fretzel.net/</id>
  <author>
    <name><![CDATA[Kent Holloway]]></name>
    
  </author>
  <generator uri="http://octopress.org/">Octopress</generator>

  
  <entry>
    <title type="html"><![CDATA[Resize Qcow2 Images Without Virt-resize]]></title>
    <link href="http://blog.fretzel.net/blog/2014/10/08/resize-qcow2-images-without-virt-resize/"/>
    <updated>2014-10-08T20:52:54-05:00</updated>
    <id>http://blog.fretzel.net/blog/2014/10/08/resize-qcow2-images-without-virt-resize</id>
    <content type="html"><![CDATA[<p>I&rsquo;m in the process of switching from kickstart builds for new VM&rsquo;s to image based clones.
I needed to find a simple way to add space to a qcow2 image file along with one of the partitions in it and the volumes it contained.
Most of the guides I found used <em>virt-resize</em> to resize the image.
While <em>virt-resize</em> makes resizing qcow2 images really easy it has the side affect of making sparse images into non-spare images.
This can end up using a considerable amount of space depending on how much additional space you are adding.
The rest of the options I found have you converting qcow2 to raw and then back again which is slow and also can consume large amounts of space.</p>

<p><a href="http://libguestfs.org/guestfs.3.html">Libguestfs</a> to the rescue!</p>

<p>Now there are a few requirements, namely this only works if the partition you want to modify is either the last partition or the only partition and you need to do this all while the VM is shutdown.
In my case it&rsquo;s the last partition I need to expand, the disk layout has boot at partition 1 and partition 2 contains two LVM volumes.
It&rsquo;s both of these volumes that I want to expand via libguestfs.</p>

<p><a href="https://gist.github.com/kholloway/ded725ea321ce8fe79c7">full gist available here</a> with some error checking and a few more options.</p>

<h4>Lets go!</h4>

<p>First lets increase the qcow2 image size using the standard <em>qemu-img</em> tool, lets add 40GB.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'>qemu-img resize vmimage.img +40G
</span></code></pre></td></tr></table></div></figure>


<p>Now using whatever language you like (libguestfs has lots of bindings for various languages) fire up the guestfs back end.
Note that you can do this in the shell/Bash if you like but calculating the proper partition sector start location is a little harder (not impossible just a bit more work).
Everything below is in Python, copy it to a script or just type along in the Python command line.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="kn">import</span> <span class="nn">guestfs</span>
</span><span class='line'>
</span><span class='line'><span class="c"># For return values we want a dict object</span>
</span><span class='line'><span class="n">g</span> <span class="o">=</span> <span class="n">guestfs</span><span class="o">.</span><span class="n">GuestFS</span><span class="p">(</span><span class="n">python_return_dict</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
</span><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">add_drive_opts</span><span class="p">(</span><span class="s">&#39;vmimage.img&#39;</span><span class="p">)</span>
</span><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">launch</span><span class="p">()</span>
</span></code></pre></td></tr></table></div></figure>


<p>Lets assume your dev name is <strong>/dev/sda</strong> but you can use the <em>list_devices</em> function to find it if you are not sure.
Now lets get the partition list:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">partitions</span> <span class="o">=</span> <span class="n">g</span><span class="o">.</span><span class="n">part_list</span><span class="p">(</span><span class="s">&#39;/dev/sda&#39;</span><span class="p">)</span>
</span></code></pre></td></tr></table></div></figure>


<p>We need the start location of partition 2:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">part_start</span> <span class="o">=</span> <span class="n">partitions</span><span class="p">[</span><span class="mi">1</span><span class="p">][</span><span class="s">&#39;part_start&#39;</span><span class="p">]</span>
</span></code></pre></td></tr></table></div></figure>


<p>Grab the block size of the image:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">blk_size</span> <span class="o">=</span> <span class="n">g</span><span class="o">.</span><span class="n">blockdev_getss</span><span class="p">(</span><span class="s">&#39;/dev/sda&#39;</span><span class="p">)</span>
</span></code></pre></td></tr></table></div></figure>


<p>Calculate the starting sector by dividing our partition start location by the block size:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">start_sector</span> <span class="o">=</span> <span class="n">part_start</span> <span class="o">/</span> <span class="n">blk_size</span>
</span></code></pre></td></tr></table></div></figure>


<p>Now the scary part, we need to delete partition 2:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">part_del</span><span class="p">(</span><span class="s">&#39;/dev/sda&#39;</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
</span></code></pre></td></tr></table></div></figure>


<p>Ok great the partition is gone, lets create a new one now with the info we gathered above.
The &lsquo;-1&rsquo; bit below tells libguestfs that you want the last sector as your end sector.
This is a nice shortcut so you don&rsquo;t need to calculate it yourself:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="k">try</span><span class="p">:</span>
</span><span class='line'>  <span class="n">g</span><span class="o">.</span><span class="n">part_add</span><span class="p">(</span><span class="s">&#39;/dev/sda&#39;</span><span class="p">,</span> <span class="s">&#39;p&#39;</span><span class="p">,</span> <span class="n">start_sector</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">)</span>
</span><span class='line'><span class="k">except</span> <span class="ne">RuntimeError</span> <span class="k">as</span> <span class="n">msg</span><span class="p">:</span>
</span><span class='line'>  <span class="k">print</span> <span class="s">&quot;Part add failed due to re-read of partition table, this is normal and expected.. Ignored..&quot;</span>
</span></code></pre></td></tr></table></div></figure>


<p>As noted above because the partition table can&rsquo;t be re-read by the kernel you will get an error message.
This is ok, to resolve this lets just stop and restart the libguestfs system to force it to reload the partition table:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">shutdown</span><span class="p">()</span>
</span><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">add_drive_opts</span><span class="p">(</span><span class="s">&#39;vmimage.img&#39;</span><span class="p">)</span>
</span><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">launch</span><span class="p">()</span>
</span></code></pre></td></tr></table></div></figure>


<p>Ok now we can resize the PV inside the partition we just expanded:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">pvresize</span><span class="p">(</span><span class="s">&#39;/dev/sda2&#39;</span><span class="p">)</span>
</span></code></pre></td></tr></table></div></figure>


<p>Now in this case I know the exact volume names but if you didn&rsquo;t know it you can use the <em>lvs</em> function in libguestfs to list them.
Since I know them already I&rsquo;m just going to include it in the command below which takes my swap volume from whatever it was (2GB previously) to 6GB now.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">lvresize</span><span class="p">(</span><span class="s">&#39;/dev/vgroot/swap.vol&#39;</span><span class="p">,</span> <span class="mi">6144</span><span class="p">)</span>
</span></code></pre></td></tr></table></div></figure>


<p>Ok great, swap vol is now 6GB!
Lets expand the root volume with any remaining space in the PV.
This uses a slightly different guestfs command that fills free space with a percentage number.
For my particular use case I just want to fill all remaining space or 100% remaining space as shown below:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">lvresize_free</span><span class="p">(</span><span class="s">&#39;/dev/vgroot/root.fs&#39;</span><span class="p">,</span> <span class="mi">100</span><span class="p">)</span>
</span></code></pre></td></tr></table></div></figure>


<p>Last steps, lets resize the actual filesystem on the root volume and then check it to make sure everything is OK:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">resize2fs</span><span class="p">(</span><span class="s">&#39;/dev/vgroot/root.fs&#39;</span><span class="p">)</span>
</span><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">e2fsck</span><span class="p">(</span><span class="s">&#39;/dev/vgroot/root.fs&#39;</span><span class="p">,</span> <span class="n">correct</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
</span></code></pre></td></tr></table></div></figure>


<p>Now shutdown the guestfs process since we are all done:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">shutdown</span><span class="p">()</span>
</span><span class='line'><span class="n">g</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</span></code></pre></td></tr></table></div></figure>


<p>That&rsquo;s it!
Looks hard but it&rsquo;s pretty simple, the hardest part is calulating your proper start sector.
Now go boot up your VM and you will find that swap has increased to 6GB and your root volume has an additional 34GB of space in it!</p>

<p>This all happens in less than a minute if you script/automate it all and doesn&rsquo;t have any of the conversion or sparse to non-sparse issues that virt-resize has.</p>

<p>Enjoy!</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Vundle for Vim]]></title>
    <link href="http://blog.fretzel.net/blog/2014/06/08/vundle-for-vim/"/>
    <updated>2014-06-08T20:08:14-05:00</updated>
    <id>http://blog.fretzel.net/blog/2014/06/08/vundle-for-vim</id>
    <content type="html"><![CDATA[<p>I have a new favorite Vim plugin manager called <a href="https://github.com/gmarik/Vundle.vim">Vundle</a>.</p>

<p>It&rsquo;s simple and doesn&rsquo;t have all the overhead of some of the other Vim plugin managers/pref bundles.</p>

<p>I&rsquo;m only using a small handful of plugins (shown below)</p>

<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>Bundle 'SuperTab'
</span><span class='line'>
</span><span class='line'>" Repos from GitHub
</span><span class='line'>Bundle 'bling/vim-airline'
</span><span class='line'>Bundle 'tpope/vim-fugitive'
</span><span class='line'>Bundle 'davidhalter/jedi-vim'
</span><span class='line'>Bundle 'rodjek/vim-puppet'
</span><span class='line'>Bundle 'elzr/vim-json'
</span><span class='line'>
</span><span class='line'>" installed bundles go here
</span><span class='line'>Bundle 'Syntastic'</span></code></pre></td></tr></table></div></figure>


<p>Vundle has been going through some upgrades lately which is making it even easier to use.</p>

<p>Take 10 minutes and check it out.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[New Blog Home and Software]]></title>
    <link href="http://blog.fretzel.net/blog/2014/06/08/new-blog-home-and-software/"/>
    <updated>2014-06-08T19:56:51-05:00</updated>
    <id>http://blog.fretzel.net/blog/2014/06/08/new-blog-home-and-software</id>
    <content type="html"><![CDATA[<p>Welcome to the new home for my little blog.
I&rsquo;m using <a href="http://octopress.org">Octopress</a> and really liking it so far.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Audio Fingerprinting on Raspberry Pi (Part 1)]]></title>
    <link href="http://blog.fretzel.net/blog/2014/03/05/audio-fingerprinting-on-raspberry-pi-part-1/"/>
    <updated>2014-03-05T17:15:43-06:00</updated>
    <id>http://blog.fretzel.net/blog/2014/03/05/audio-fingerprinting-on-raspberry-pi-part-1</id>
    <content type="html"><![CDATA[<p>A short guide on how to do some basic audio fingerprinting on a Raspberry Pi also known as “my quest to notify myself when the dryer is done”.
Lets start with the excellent Chromaprint library.
Build Chromaprint on Pi</p>

<p>Install pre-reqs:</p>

<pre><code>sudo apt-get install libboost1.50-all-dev ffmpeg libtag1-dev zlib1g-dev resample libresample1-dev cmake libffms2-dev  
</code></pre>

<p>Download package from Chromaprint source</p>

<pre><code>mkdir /opt/src
cd /opt/src
wget https://bitbucket.org/acoustid/chromaprint/downloads/chromaprint-1.1.tar.gz
tar -zxvf chromaprint-1.1.tar.gz
cd chromaprint\*
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_EXAMPLES=OFF -DCMAKE_CXX_FLAGS="-Ofast -mfpu=vfp -mfloat-abi=hard -march=armv6zk -mtune=arm1176jzf-s" -DBUILD_TOOLS=ON
make
sudo make install
</code></pre>

<p>Python stuff</p>

<p>If you don’t already have PIP installed go grab it:</p>

<pre><code>sudo easy_install pip
</code></pre>

<p>Build/install Python packages now:</p>

<pre><code>sudo pip install pyacoustid audioread
</code></pre>

<p>Create a test WAV file 10 seconds in length of whatever audio sound/noise/whatever that you want to fingerprint.
In this case I’m using my USB microphone plugged into the Pi since there is no onboard microphone or audio input (and no analog inputs either).
You can also use the PyAudio package you installed to record the wav file.</p>

<pre><code>arecord -D plughw:1,0 -d 10 -r 44100 --channels=2 --format=cd /tmp/test.wav
</code></pre>

<p>Fingerprint your wave file (in python). Note: You can just as easily use ‘fpcalc’ in the shell to get a compressed or raw fingerprint from chromaprint libs.</p>

<pre><code>import chromaprint
import acoustid
duration, fingerprint = acoustid.fingerprint_file('/tmp/test.wav')
fp_raw = chromaprint.decode_fingerprint(fingerprint)[0]
print "Compressed fingerprint: %s" % fingerprint
print "Raw fingerprint: %s" % fp_raw
</code></pre>

<p>If all went well you should not have seen any errors and your compressed and raw fingerprint should have been printed to the screen.
If you fingerprint only had “AQAA” in it then something is wrong with your chromaprint library.
The available libchromaprint packages along with the python-pyaudio package did not work for me at all and consistently resulted in no fingerprints.
Instead building the custom chromaprint libs and then installing pyaudio via pip resulted in a working setup.</p>

<p>If you do have fingerprint errors make sure your wav file is not empty and if it’s not empty then try to do a clean build on the chromprint libs and re-install.</p>

<p>Note: The example ‘fpcalc’ won’t actually build from the 1.1 chromaprint libs. It throws some strange compiler errors that I couldn’t resolve.</p>

<p>I’m going to be recording 2 10 second files in a loop and that can be a bit hard on the SD card in the Raspberry Pi so instead I’m using the built in TMPFS location in /run so I don’t do a lot of unnecessary writes to the SD card.</p>

<p>That’s all for now, come back for part 2 where I wrap it all up into a single script that records in a thread while analyzing the previous recording and sending notifications when matches are found.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Multi-master Puppet Setup With DNS SRV Records]]></title>
    <link href="http://blog.fretzel.net/blog/2014/02/21/multi-master-puppet-setup-with-dns-srv-records/"/>
    <updated>2014-02-21T21:00:00-06:00</updated>
    <id>http://blog.fretzel.net/blog/2014/02/21/multi-master-puppet-setup-with-dns-srv-records</id>
    <content type="html"><![CDATA[<p>I recently switched from a single Puppet master to a Multi-master setup which consists of a single CA server and 2 new masters.
During this change I also took the time to upgrade my clients (and server) from 2.7.25 to 3.4.2 and Ruby from 1.8.7 to 2.0.0p353 oh and I also switched over to the new DNS SRV records setup.</p>

<p>While I&rsquo;m not going to rehash the entire setup and all steps taken to get there I do want to include some high level steps that were not exactly clear after reading the Puppet labs docs on multi-master setups. Hopefully this helps others trying to accomplish the same thing.</p>

<p>For my setup I used a standalone CA and 2 masters.  Follow the Puppet labs guides/docs to build out your CA and masters with whatever software you like. I used the blessed Apache + Passenger setup. For multi-masters there is no special setup required on the Apache/Passenger side of things, just set them up as usual with the exception of your config.ru file (see below).</p>

<p>If you are upgrading from a single master 2.X setup you also need to remove any $servername references from your manifests. Most likely this will be in manifests/site.pp file.</p>

<p>Don&rsquo;t just copy your config.ru from your old setup if you ran Puppet 3.x or older.
Use the new config.ru.passenger.3 in the Puppet labs github repo. If you skip this step you will have a series of odd problems that you won&rsquo;t be able to resolve any other way. Make sure you chown the config.ru file as puppet:puppet since Passenger uses the owner of the file as the user to run as.</p>

<p>Before you start your new CA or master servers you have to generate the SSL certs properly.
On the CA: make sure your /etc/puppet/puppet.conf contains the lines below (adjust config as needed to support your setup):</p>

<pre><code> [main]
  pluginsource = puppet:///plugins
  pluginsync = true
  use_srv_records = true
  srv_domain = mydomain.com

 [master]
  ca = true
  dns_alt_names=myca1.mydomain.com,myca1
  # Bits for Passenger/Apache
  certname=puppetca.mydomain.com
  ssl_client_header=SSL_CLIENT_S_DN
  ssl_client_verify_header=SSL_CLIENT_VERIFY
</code></pre>

<p>Now run the command below on your CA to generate your CA certs with the proper dns_alt_names.
Puppetca is an alias pointing at my hosts real name, DNS alt names should contain your hosts real name.</p>

<pre><code>puppet cert generate puppetca.mydomain.com --dns_alt_names=myca1.mydomain.com,myca1
</code></pre>

<p>Verify that your cert looks correct with the command below, it should list your puppetca plus the alternate DNS names you specified.</p>

<pre><code>puppet cert list puppetca.mydomain.com
</code></pre>

<p>Your CA is now ready to run, fire up the web server and double check your weblogs for any errors. Assuming all is good now you can switch over to one of your masters and make sure your config contains the bits below. The really important line is ca=false for any server that is not your CA server.</p>

<pre><code>[main]
   pluginsource = puppet:///plugins
   pluginsync = true
   use_srv_records = true
   srv_domain = mydomain.com

[master]
   ca = false
   # Bits for Passenger/Apache
   certname=master1.mydomain.com
   ssl_client_header=SSL_CLIENT_S_DN
   ssl_client_verify_header=SSL_CLIENT_VERIFY
</code></pre>

<p>Run your master by hand the first time:</p>

<pre><code>puppet master --no-daemonize --verbose
</code></pre>

<p>The master will generate it&rsquo;s cert and send it over to the CA server to get signed. If you are using autosigning just wait for the cert to be signed, if not go sign it on the CA server.</p>

<p>Once that cert is signed you can hit CTRL-C and stop your master, now start it back up using the real web service. Once again check the weblogs for any errors. Try running the puppet agent by hand on this master now and see how it goes. You should get a clean run.</p>

<p>Now head over to your 2nd or 3rd master and repeat the steps above for the masters.</p>

<p>With your masters and your CA server working you can now tackle the clients.
Using your existing puppet master (if you have one) add all the lines in the [main] section above to your clients. You can safely do this ahead of time because the 2.X clients don&rsquo;t support those features and will just ignore them.</p>

<p>Now upgrade your packages via whatever tools you use to do package upgrades, for my setup I have a custom build of Ruby 2.0 packaged as an RPM using a fairly standard SPEC file.
I then used the FPM utility to package up Puppet, Facter and all dependancies (don&rsquo;t forget about Augeas if you use it).</p>

<p>Now on my hosts I can do a &lsquo;yum install ruby20-puppet&rsquo; and everything gets upgraded.
Make sure your Puppet.conf file has those srv_domain bits above and then delete your clients &lsquo;ssl&rsquo; directory.
Run the agent, it should automatically switch over to the new CA and masters and generate a cert, go sign it (or turn on autosigning), once signed the client should finish it&rsquo;s run as usual.</p>

<p>One final note: Currently Puppet pluginsync is broken with 3.4.2 (and below) when using DNS SRV records. This should be fixed in a later version but the simple workaround for now is to remove the implied $servername portion in pluginsync and instead let it use the server that the client connected to by putting this line in each and EVERY puppet.conf file for both agents and masters in the [main] section.</p>

<pre><code> [main]
  pluginsource = puppet:///plugins
</code></pre>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Vim in All It's Awesomeness]]></title>
    <link href="http://blog.fretzel.net/blog/2013/12/05/vim-in-all-its-awesomeness/"/>
    <updated>2013-12-05T07:36:29-06:00</updated>
    <id>http://blog.fretzel.net/blog/2013/12/05/vim-in-all-its-awesomeness</id>
    <content type="html"><![CDATA[<p>I have tried many text editors (and various IDE&rsquo;s) over the years and I always come back to VIM for just about everything, especially anything programming related.
Vim on it&rsquo;s own is great but with some plugins and some simple changes it&rsquo;s awesome.
Everything from syntax highlighting (using Syntastic) to autocompletion of functions and variable names.
Since I write a lot of code in Python and Puppet (Syntastic has checkers for both and much more) which has saved me an incredible amount of time since I started using it.</p>

<p>Do yourself a favor and take 5 minutes to go get the incredible <a href="https://github.com/carlhuda/janus">Janus: Vim Distribution</a>.
It&rsquo;s a quick and painless install.
From the Janus README:</p>

<pre><code>$ brew install macvim    (optional - requires [Homebrew](http://brew.sh/))
$ curl -Lo- https://bit.ly/janus-bootstrap | bash
</code></pre>

<p>The above two commands are all you need, that second command will also backup your existing Vim files in your home directory so you don&rsquo;t lose anything you may already have setup.</p>

<p>Customization is fairly simple using ~/.vimrc.before and ~/.vimrc.after files in your home dir.
The only changes I make to the base Janus setup is shown below, put these changes in ~/.vimrc.after if you like.</p>

<pre><code>" Clear searches easily with ,/ after
nmap &lt;silent&gt; ,/ :nohlsearch&lt;CR&gt;

" Give a shortcut key to NERD Tree
map &lt;F2&gt; :NERDTreeToggle&lt;CR&gt;

" Disable F1 help crap, map to ESC instead
map &lt;F1&gt; &lt;Esc&gt;
imap &lt;F1&gt; &lt;Esc&gt;
</code></pre>

<p>One of my favorite commands once you have Janus installed is to reformat an entire file with a simple keystroke:</p>

<ul>
<li>\&lt;leader>fef formats the entire file</li>
</ul>


<p>The default \&lt;leader> character is \ so \fef reformats your current file.
Stop living in the dark ages, go install Janus right now!</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Git - Move Subdirectory to New Repo]]></title>
    <link href="http://blog.fretzel.net/blog/2013/04/04/git-move-subdirectory-to-new-repo/"/>
    <updated>2013-04-04T15:08:22-05:00</updated>
    <id>http://blog.fretzel.net/blog/2013/04/04/git-move-subdirectory-to-new-repo</id>
    <content type="html"><![CDATA[<p>I had a need to move/detach a subdirectory that was inside a larger Git repository into it&rsquo;s own smaller/standalone repository.
After a few Google searches it turns out this is a fairly common and relatively easy thing to do.</p>

<p>Not required but I like to start with a fresh clone of the repo I&rsquo;m working with into a temp directory.</p>

<pre><code>mkdir tmp
cd tmp
git clone my_original_repo_url
</code></pre>

<p>Now clone the repo again (this time it&rsquo;s a local only clone of the repo above):</p>

<pre><code>git clone --no-hardlinks my_original_repo new_repo_name
cd new_repo_name
</code></pre>

<p>Extract just the subdirectory you want:</p>

<pre><code>git filter-branch --subdirectory-filter mysubdir
</code></pre>

<p>Now lets remove the old remotes, any unneeded history and repack the repo:</p>

<pre><code>git remote rm origin
git update-ref -d refs/original/refs/heads/master
git reflog expire --expire=now --all
git repack -ad
</code></pre>

<p>Now you can add your new remote(s) in and push your changes up to the server:</p>

<pre><code>git remote add origin my_new_repo_url
git push origin master
</code></pre>

<p>Note: If you are using Gitorious it may at this point complain about a &lsquo;invalid ref&rsquo; when you push it to the server.
As far as I can tell this does not cause any problems and only occurs on the first push.</p>

<p>So that covers making your new repo from a subdirectory now lets go remove the now old subdirectory from the original repo so we don&rsquo;t commit to it by accident.
I&rsquo;m using a simplified removal process, you could remove all references and commit info for the subdirectory if you like but for my case that was overkill.</p>

<pre><code>cd ../my_original_repo
rm -rf mysubdir
git rm -r mysubdir
git commit -m "Removing subdir, it has been moved to its own repo now"
git push origin master
</code></pre>

<p>All done!
Your subdirectory has now been moved from the original repository into a new repository with all your history and commits intact.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Gitorious/Redis Installer Updated]]></title>
    <link href="http://blog.fretzel.net/blog/2013/03/26/gitorious-slash-redis-installer-updated/"/>
    <updated>2013-03-26T16:40:00-05:00</updated>
    <id>http://blog.fretzel.net/blog/2013/03/26/gitorious-slash-redis-installer-updated</id>
    <content type="html"><![CDATA[<p>In regards to my last post <a href="http://blog.fretzel.net/blog/2013/03/25/gitorious-and-the-redis-service/">gitorious-and-redis-service</a> I have updated my clone of the ce-installer to include the changes up to v2.4.12 and rolled all my changes from that post into the latest version.</p>

<p>My CE-Installer clone:
 <a href="https://www.gitorious.org/~kholloway/gitorious/kholloways-ce-installer">https://www.gitorious.org/~kholloway/gitorious/kholloways-ce-installer</a></p>

<p>I have requested a merge with the mainline ce-installer which is viewable at:
 <a href="https://www.gitorious.org/gitorious/ce-installer/merge_requests/2">https://www.gitorious.org/gitorious/ce-installer/merge_requests/2</a></p>

<p>Enjoy!</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Gitorious and the Redis Service]]></title>
    <link href="http://blog.fretzel.net/blog/2013/03/25/gitorious-and-the-redis-service/"/>
    <updated>2013-03-25T15:03:09-05:00</updated>
    <id>http://blog.fretzel.net/blog/2013/03/25/gitorious-and-the-redis-service</id>
    <content type="html"><![CDATA[<p>Gitorious has a very nice status command via /usr/bin/gitorious_status that quickly shows you if all your Gitorious services are up and running (see screenshot).</p>

<p><img class="left" src="http://blog.fretzel.net/images/health_check.png" width="300" height="250"></p>

<p>It&rsquo;s missing one very important service though, Redis!</p>

<p>Redis is now the default messaging service when you do a fresh install via the <a href="http://getgitorious.com/installer">Gitorious CE-Installer</a> but it&rsquo;s not included in the status check, it&rsquo;s missing from the /admin/diagnostics page and it&rsquo;s missing a <a href="http://mmonit.com/monit/">Monit</a> check file to restart it if it dies. Keeping it running is pretty important for a working Gitorious install because without it many of the web page operations like creating a new project or team will fail and it won&rsquo;t be very clear from the logs why it failed.</p>

<p>Lets fix some of those problems.</p>

<p>First off lets patch the gitorious_status script with the patch below which should work on any modern Linux variant.</p>

<p>Save the lines below as: <strong>/tmp/my.patch</strong></p>

<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
<span class='line-number'>16</span>
<span class='line-number'>17</span>
<span class='line-number'>18</span>
<span class='line-number'>19</span>
<span class='line-number'>20</span>
<span class='line-number'>21</span>
<span class='line-number'>22</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>*** gitorious_status 2013-03-25 16:11:04.475121039 -0500
</span><span class='line'>--- gitorious_status_redis 2013-03-25 16:16:35.380135982 -0500
</span><span class='line'>*************** unicorn_status() {
</span><span class='line'>*** 26,31 ****
</span><span class='line'>--- 26,35 ----
</span><span class='line'>      check_process_and_report "ps -p $PID" "Unicorn"
</span><span class='line'>  }
</span><span class='line'>
</span><span class='line'>+ redis_status() {
</span><span class='line'>+     check_process_and_report "/etc/init.d/redis status" "Redis"
</span><span class='line'>+ }
</span><span class='line'>+
</span><span class='line'>  # Upstart's exit codes are a beast of its own
</span><span class='line'>  resque_status() {
</span><span class='line'>      check_process_and_report "/sbin/initctl status resque-worker" "Resque"
</span><span class='line'>*************** sphinx_status
</span><span class='line'>*** 80,82 ****
</span><span class='line'>--- 84,87 ----
</span><span class='line'>  memcached_status
</span><span class='line'>  sshd_status
</span><span class='line'>  mysqld_status
</span><span class='line'>+ redis_status
</span></code></pre></td></tr></table></div></figure>


<p>Apply your patch to the status command:</p>

<pre><code> patch /usr/bin/gitorious_status &lt; /tmp/my.patch
</code></pre>

<p>Your status command should now show the details of the Redis service as shown below.</p>

<p><img class="left" src="http://blog.fretzel.net/images/redis-status.png" width="300" height="250"></p>

<p>Next lets create a Monit config file for Redis which will watch the process and restart if needed.
I use Puppet and a custom Monit module I wrote for this but it&rsquo;s not required.
For now lets just manually create the file and you can integrate it into your configuration management tool later if you like. Note that the Monit file below is specific to Redhat, change the pidfile and start/stop lines as needed to match your OS.</p>

<p>Copy the contents below into:
   <strong>/etc/monit.d/redis.monit</strong></p>

<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>check process redis-server with pidfile /var/run/redis/redis.pid
</span><span class='line'>  start program = "/sbin/service redis start"
</span><span class='line'>  stop program = "/sbin/service redis stop"
</span><span class='line'>  if does not exist for 1 cycles then restart
</span><span class='line'>  if 5 restarts within 5 cycles then alert</span></code></pre></td></tr></table></div></figure>


<p>Now lets restart Monit so it picks up the change.
On Redhat that&rsquo;s done like so:</p>

<pre><code>service monit restart
</code></pre>

<p>Check that it&rsquo;s setup correctly in Monit:</p>

<pre><code>monit summary
</code></pre>

<p>OR</p>

<pre><code>monit status
</code></pre>

<p>So now you have Redis monitored by Monit and the gitorious_status command shows you if it&rsquo;s up or down but your /admin/diagnostics page is still missing any status about it. That last bit is not too hard to fix but for now I&rsquo;m leaving that up to the Gitorious folks to patch along with the incorrect status about the gitorious-poller service which is not in use any longer.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[My New RSS Setup After the Death of Google Reader]]></title>
    <link href="http://blog.fretzel.net/blog/2013/03/16/my-new-rss-setup-after-the-death-of-google-reader/"/>
    <updated>2013-03-16T10:25:00-05:00</updated>
    <id>http://blog.fretzel.net/blog/2013/03/16/my-new-rss-setup-after-the-death-of-google-reader</id>
    <content type="html"><![CDATA[<p>My setup for now, so far working very well.</p>

<p><a href="http://www.feedafever.com/">FeedaFever</a> on my server (I love owning my own data!)
Reeder iOS app on the iPhone and I&rsquo;m hoping the Reeder dev adds FeedaFever support for Reeder on iPad and Mac soon.</p>

<p>Fever has a usable web interface on both iPad and iPhone if you don&rsquo;t have a native app but Reeder is the gold standard on iOS for an RSS reader.</p>

<p>Alternative iPhone app choice for FeedaFever: <a href="https://itunes.apple.com/us/app/sunstroke/id488564806?mt=8">Sunstroke</a></p>

<p>Some alternatives I looked at and still might use down the road are <a href="http://www.newsblur.com/">NewsBlur</a> and <a href="http://www.feedly.com/">Feedly</a>.
NewsBlur is hosted with free accounts or premium accounts and if you want it&rsquo;s open source so you can download and run your own server.</p>

<p>Feedly looks nice also but I&rsquo;m not used to the layout/setup yet.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Git CE Installer for RHEL 6.X]]></title>
    <link href="http://blog.fretzel.net/blog/2013/03/04/git-ce-installer-for-rhel-6-dot-x/"/>
    <updated>2013-03-04T07:36:51-06:00</updated>
    <id>http://blog.fretzel.net/blog/2013/03/04/git-ce-installer-for-rhel-6-dot-x</id>
    <content type="html"><![CDATA[<p>My small modifications to the main Gitorious CE installer so it installs cleanly on Redhat/RHEL 6.X instead of just on CentOS 6.X.
It also does not try to update Puppet when it&rsquo;s already installed.</p>

<p>I&rsquo;m maintaining a Git repo of my changes at Gitorious:
 <a href="https://gitorious.org/~kholloway/gitorious/kholloways-ce-installer">Custom Gitorious CE Installer for RHEL</a></p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[SVN to Git]]></title>
    <link href="http://blog.fretzel.net/blog/2013/03/04/svn-to-git/"/>
    <updated>2013-03-04T07:23:58-06:00</updated>
    <id>http://blog.fretzel.net/blog/2013/03/04/svn-to-git</id>
    <content type="html"><![CDATA[<p>SVN -> Git:</p>

<p> Good post by Makis about how to setup SVN to Git conversions.</p>

<p><a href="http://blog.sofistes.net/2010/07/transferring-svn-repository-to.html">Transferring-svn-repository-to-git</a></p>
]]></content>
  </entry>
  
</feed>