Unconventional tools, #2

Hi All;

Now, we’ve all seen code that at least appears to work, but is very poorly written. You might even say the code “stinks”. Now, it can be really hard to detect code smells in your own code. A major air freshener company speaks of people going “noseblind” to smells. The old solution to this problem was to have code reviews. This never works. People gloss over problems, the reviews take forever, and feelings can be hurt or axes can be ground. What is needed is an objective, easy-to-perform, review of the code. What is needed is an automated tool!

For me, programming in free ruby, the tool is called… reek!

Now you may think: “Why do I need a tool to complain about my working code?” I must admit, I sometimes think that too. The answer to that question is that whenever I dig into the code and clean up the smells, the result is almost always much better code. Now, I know that’s a serious claim, so I am going to back it up with a real example from my own code. The following bit of code is used to process snippets of ruby code embedded in strings and surrounded by {{  }} structures. They generally go by the name “handlebars”. Here is our starting point:

#Process a string with code embedded in handlebars and 
#backslash quotes.
def eval_handlebars(in_str)
  out_str = ""

  loop do
    pre_match, match, in_str = in_str.partition(/{{.*?}}/m)
    out_str << pre_match
    return out_str.gsub(/\\\S/) {|found| found[1]} if match.empty?

    code = match[2...-2]
    silent = code.end_with?("#")
    result = instance_eval(code)
    out_str << result.to_s unless silent

 When I ran reek against this code, it made the following observations:

lib/mysh/user_input/handlebars.rb -- 2 warnings:
 [25, 27, 32]:FeatureEnvy: Object#eval_handlebars refers to
  out_str more than self (maybe move it to another class?)
 [19]:TooManyStatements: Object#eval_handlebars has approx 10 
2 total warnings

Well, the code is pretty ugly, so perhaps this is not surprising. So; how to proceed? The first clue comes from the comment line.

#Process a string with code embedded in handlebars and 
#backslash quotes.

This method is doing two distinct things! Maybe it should be two distinct methods? So here’s the first fixup:

#Process a string with code embedded in handlebars and 
#backslash quotes.
def eval_handlebars(str)
  do_process_handlebars(str).gsub(/\\\S/) {|found| found[1]}


#Process a string with code embedded in handlebars.
def do_process_handlebars(in_str)
  out_str = ""

  loop do
    pre_match, match, in_str = in_str.partition(/{{.*?}}/m)
    out_str << pre_match
    return out_str if match.empty?

    code = match[2...-2]
    silent = code.end_with?("#")
    result = instance_eval(code)
    out_str << result.to_s unless silent

The new method do_process_handlebars just does handlebars while the original method takes that result and processes backslash quotes. So what does reek think of the code now?

lib/mysh/user_input/handlebars.rb -- 1 warning:
 [26]:TooManyStatements: Object#do_process_handlebars has approx 9 
1 total warning

This is a lot better, but now my new method is too long! What to do? I could ignore the problem; I could tell reek to just ignore the problem; or I could mashup the code and use sneaky tricks to make it use fewer lines. None of these are good choices here. Instead, let’s really look at what the code is doing. We see a loop, processing a string by repeatedly looking for a regular expression and then performing a substitution of the found text with new text. Wow! Did I really write that code? What we have here is a kludgy reinvention of the gsub method! The very same method already used to perform the backslash quoting. Zounds!

OK; so let’s see what happens when we replace the kludge of gsub with that actual gsub method:

#Process a string with code embedded in handlebars.
def do_process_handlebars(str)
  str.gsub(/{{.*?}}/m) do |match|
    code = match[2...-2]
    silent = code.end_with?("#")
    result = instance_eval(code)

    (result unless silent).to_s

 Wow! That code is MUCH better looking! What does reek think about it?

0 total warnings

No more code smells found, well at least by the reek tool. I did make one further change. I corrected the ambiguous top comment line:

#Process a string with backslash quotes and code embedded 
#in handlebars.

It would seem that even automated code scanning tools do not check for poorly written comments.

So, I think it is pretty clear that the new code is much better than the original. I can tell you that it also runs faster and uses less memory. Can we draw a conclusion from all of this? How about:

When things smell bad, put away the scented air spray and just clean up the mess!

Best regards;

Peter Camilleri (aka Squidly Jones)


  1. In the quest to write better code, I am inspired by Sandi Metz. An awesome video by her on the matter of smelly code is Get a Whiff of This by Sandi Metz.
  2. Some code was slightly reformatted to fit into the blog post.





The best news I’ve heard in a LONG time!

Hello All!

One of the most pernicious and damaging mistakes, injurious to world progress, has to have been the decision to allow the patenting of software. These patents have served to obstruct progress, destroy innovation, and fatten the wallets of lawyers and greedy patent trolls.

Now, to some, software patents are an awesome idea. If your goal is to provide more work for legal departments at large companies or you need to attack a competitor with the gaul of being better than you, then software patents are for you! They’re also great at destroying small, upstart companies with all of their “new ideas” stuff. Heck, even if the patent does NOT apply, prolonged legal proceedings can serve just as well.

Seriously, software patents are of dubious virtue at the best of times. Now finally, after many years of suffering under the lawyer’s savage yoke, there may be some hope:

Here’s Why Software Patents Are in Peril After the Intellectual Ventures Ruling

This a great article with a video for the tl;dr; crowd. Still, I’d like to focus on a little excerpt from the article that is, I think, the heart of the matter:

Pointing out that intellectual property monopolies can limit free speech, Mayer notes that copyright law has built-in First Amendment protections such as “fair use” and that patent law must include similar safeguards. He suggests that the safeguard comes in the form of a part of the Patent Act, known as “Section 101,” which says some things—including abstract ideas—simply can’t be patented in the first place.

I really hope that this is the beginning of the end for software patents! I really do! I look forward to the day when they and the profits of the trolls are all dead and buried. Just don’t expect me to bring flowers to the grave site.

Best regards;

Peter Camilleri (aka Squidly Jones)


Gem Download Study: 24 Weeks

Well 12 more weeks has passed and it is time for the next installment of the Ruby Gems Download study. The goal of this study was to see if it was possible to observe patterns in the download rates that might lead to useful conclusions about the mix of entities doing the downloading. The gems repository makes this download data readily available. For my gems, you can see that data here.

For the 24 week report there are two main data findings. A graph of individual, cumulative downloads for each gem for 24 weeks and a graph of the weekly downloads for all gems for the same period.


I have collected 24 weeks of data, so here is the graph showing the rate of downloads.

If you look, you will see that the slopes of the lines vary. Some lines are very flat, while others are sloped upward at a much sharper angle. This means that the rate of downloads is also different.


The weekly, combined results show that the rate of downloads is anything but constant. The valleys seem to correspond to periods this summer when large numbers of people would be on vacation (Not me of course; I’m never/always on vacation)

The spiky peaks do tend to correspond to times when large numbers of gems were mass updated. This is especially the case for the last and fourth from last points where a code of conduct and changes to the mini_readline gem were propagated to most of the gems. This is clearly indicative of automated downloading.

This study shall continue. I estimate that the next posting on this matter will be one to study an entire year’s worth of data.

Until then, Many Thanks and Best Regards;

Peter Camilleri (aka Squidly Jones)

Update: fOOrth 0.6.5 released

Hello Again!

This release of fOOrth continues to work its way through the backlog of work on the way to version 0.7.0 and the ultimate goal of version 1.0.0. Here is a brief summary of what’s new in this release:

Updated for V0.6.5

  • Added the ability to override the default method mapping.
  • Added report formatting to allow arrays and hashes to be displayed as bullet points and arrays as neat, efficient columns.
  • Added several deep introspection methods for objects, methods, and classes.
  • Bug Fix: Instance variables may now be created by exclusive methods.
  • Cleanup: The “force compile” flag was another carry over from FORTH that had no place in fOOrth. It has finally been removed.

The code may be found on GitHub and the gem on RubyGems. Also don’t forget to checkout the fOOrth support site on http://www.foorth.org/.

The big sticking point, now under study, is the issue of defining instance variables in a procedure. This will likely require changes to the method mapping of instance variables, assuming this change is actually made.

In the mean time, best regards;

Peter Camilleri (aka Squidly Jones)

One more bit please!


 Hello All!

Well it has been some time since I began working in Ruby and creating little gems of code for my page in the the RubyGems.org repository. When I looked at my stats this morning I realised that I had reached a milestone of sorts. The total downloads counter showed a value of 65,881 downloads!

Now, it is a well known fact that a sixteen bit, unsigned integer can count up to 65,536. This means that my download total no longer fits into that data type. While this is not nearly as impressive as the video Gangnam Style blowing through the YouTube signed 32 bit integer view count limit of 2,147,483,647, it is my accomplishment! 😉

One more bit please! That’s right, I’d like a seventeen bit integer, Thank You Very Much!

Best regards;

Peter Camilleri (aka Squidly Jones)

Unconventional tools, #1

Hello all;

In this series of posts, we shall take a look at some unconventional tools that help solving various programming (and other) problems that often come up. For this first article, I want to tackle a problem that actually made the list of the two hardest problems in computers science. Here’s that list:

  1. Selecting meaningful names.
  2. Cache coherency and invalidation.
  3. One off errors.

Ignoring the one-off error in the list, the first issue is selecting meaningful names. Way back in the bad old days, it was considered acceptable to have a program full of entities like i, j, k, and pmz21. Variable names that were meaningless, supported by a mountain of comments that were often completely out of data and/or misleading.

Slowly, computers improved and space was no longer so violently cramping the writers style anymore. This however lead to the opposite problem of names like:

if (index_selection > the_number_of_selections_available_in_the_menu)

Names so verbose that they obscure the intent of the code.

What we really need is names that are meaningful and not verbose. It boils down to a matter of selecting the right words. Regardless of what programming language I use, I program in the english language. There are over a million words defined for that language and I can assure you the I do NOT know them all. For those where english is not their native tongue, the problem is even worse.

What to do? Use a tool to help navigate the possible words for the job: Thesaurus.com!

Let’s try a real example I encountered just this morning. I was looking to update the description of my mini_readline ruby gem. I wanted to describe the fact that it came with four sample auto-completing thing-a-ma-bobs. The only word that came to mind was “engine” that sort of worked but sounded to mechanical. I wanted a word that would express the idea of work done on the user’s behalf. So of to the thesaurus web site and punched in engine… A large page of results including the word I sought for: agent!

So the next time you struggle with one of the most difficult tasks in programming (and writing too) consider using a free tool to ease the burden of picking that one in a million perfect word for the job!

Best regards;

Peter Camilleri (aka Squidly Jones)

Gem Download Study: The first 12 weeks.

As most of the readers of this blog know, I release a lot of Ruby language gems. They are announced on this web site regularly. Now it is possible for me to check how often these gems are downloaded. You can see that data here.

Now fairly early on, the view was expressed that most (if not all) of downloads I was seeing were the result of “bots” that grab everything they can find and gather the results for their industrial overlords, most likely in China. For some time I’ve wondered if that was true. To be honest, I hope it isn’t, but the nagging doubt never left me.

So I came up with a plan. I would track the downloads of my gems on a weekly basis for 12 weeks and see if I could see some trends that might indicate the nature of the download traffic. Well, it’s been 12 weeks and here are my initial results:

Gem_Downloads_12Now I know it’s a bit of an eye chart, but just click on it to get a better view of the data. Maybe even open it in another tab for easier reference.

The first thing that becomes visible is that the slopes of the lines, and thus the rate of downloads, is not the same for all of the gems. Some of the gems are clearly more popular than others.

Even more telling, is that for one gem, “vls”, a change was made recently, that described the use of this gem with the popular “rails” web framework/library/system. Once this appeal to this very active group was made, the rate clearly began an sharp upward trajectory.

Now this is only 12 weeks worth of data, so one should not read too much into these results, but they seem to indicate that at least some of these downloads can be traced to real people. Time will tell. I plan on my next study release at the 24 week point.

Yours Truly

Peter Camilleri (aka Squidly Jones)

Announcing the Mystikó Encryption gem

Let me start by saying that there is some very important advice to any programmers who might be tempted to create the own encryption security systems. That advice is well summarized by the word: DON’T!

Most home-brewed encryption schemes, including many used in vital embedded systems, are laughably weak. Straying from proven, working standards is a recipe for disaster and loss. So I say again: DON’T!

Having said that, why am I now going to discuss the Mystikó encryption gem? Very simply, this gem is only intended for educational and amusement purposes. Under no circumstances should it be construed to be a serious encryption tool or fit for protecting actual property, privacy or security.

So why do it? Why not? It was a fun challenge and a good follow on for my earlier work with pseudo-random number generators. It was also an interesting way to learn a little bit more about this fascinating topic while having fun with Ruby programming.

The gem itself is at: The Mystiko gem. The source code may be found at: Source Code. The source code also comes with a special added treat! In the Challenge folder of the source code you will find the file “challenge.bin” This file contains a paragraph of famous text encrypted by the Mystiko gem.

At this time, I am still working on raising some funds/prizes for the first person to crack this code, after which the plain text and the key will also be published so that others may verify the winning entry!

So, what shall happen? Will a jury rigged encryption program stand the test of time? or will it be cracked soon as predicted by common wisdom? Time will tell!

Yours Truly

Peter Camilleri (aka Squidly Jones)

Announcing the vls utility.

One of the joys of modern programming is the ability to utilize external code libraries to speed up development, saving time, and reducing wasted effort. The Ruby language is especially blessed in this regard with its system of code gems. Instead of the narrow minded NIH (not invented here) mindset, the world has become our tool chest.

There is, however, a downside to this modular nirvana. Versioning! To be precise: Am I using the correct version of each (and every) little code gem? This problem goes back all the way to the old Visual Basic days. Back then it was called DLL Hell, as developers struggled to maintain a myriad of cryptic, often poorly documented binary files.

Now Ruby does have the bundler utility that allows gem versions to be specified, but what if you simply want to know: When I use this application, what modules/classes are being used?

The vls utility answers that question. To use this simply enter:

$ vls <names>

where names are a list of gems/files to be required before the modules are listed. Here, see an example of this in action:

$ vls fOOrth
vls (VersionLS): 0.1.0

Bignum, 0.0.5
Complex, 0.0.5
Date::Infinity, 0.0.5
FalseClass, 0.0.5
Fixnum, 0.0.5
Float, 0.0.5
FormatEngine, 0.7.2
FullClone, 0.0.5
Gem, 2.2.2
InArray, 0.1.5
Integer, 0.0.5
MiniReadline, 0.4.8
NilClass, 0.0.5
Numeric, 0.0.5
Rational, 0.0.5
Regexp, 0.0.5
RubySscanf, 0.2.1
SafeClone, 0.0.3
Symbol, 0.0.5
TrueClass, 0.0.5
XfOOrth, 0.6.1

The vls gem may be found at: https://rubygems.org/gems/vls and the source code lives at: https://github.com/PeterCamilleri/vls.

Yours Truly

Peter Camilleri (aka Squidly Jones)