Gem Download Study: 24 Weeks

Well 12 more weeks has passed and it is time for the next installment of the Ruby Gems Download study. The goal of this study was to see if it was possible to observe patterns in the download rates that might lead to useful conclusions about the mix of entities doing the downloading. The gems repository makes this download data readily available. For my gems, you can see that data here.

For the 24 week report there are two main data findings. A graph of individual, cumulative downloads for each gem for 24 weeks and a graph of the weekly downloads for all gems for the same period.

gem_downloads_24

I have collected 24 weeks of data, so here is the graph showing the rate of downloads.

If you look, you will see that the slopes of the lines vary. Some lines are very flat, while others are sloped upward at a much sharper angle. This means that the rate of downloads is also different.

weekly_downloads_24

The weekly, combined results show that the rate of downloads is anything but constant. The valleys seem to correspond to periods this summer when large numbers of people would be on vacation (Not me of course; I’m never/always on vacation)

The spiky peaks do tend to correspond to times when large numbers of gems were mass updated. This is especially the case for the last and fourth from last points where a code of conduct and changes to the mini_readline gem were propagated to most of the gems. This is clearly indicative of automated downloading.

This study shall continue. I estimate that the next posting on this matter will be one to study an entire year’s worth of data.

Until then, Many Thanks and Best Regards;

Peter Camilleri (aka Squidly Jones)

Update: fOOrth 0.6.5 released

Hello Again!

This release of fOOrth continues to work its way through the backlog of work on the way to version 0.7.0 and the ultimate goal of version 1.0.0. Here is a brief summary of what’s new in this release:

Updated for V0.6.5

  • Added the ability to override the default method mapping.
  • Added report formatting to allow arrays and hashes to be displayed as bullet points and arrays as neat, efficient columns.
  • Added several deep introspection methods for objects, methods, and classes.
  • Bug Fix: Instance variables may now be created by exclusive methods.
  • Cleanup: The “force compile” flag was another carry over from FORTH that had no place in fOOrth. It has finally been removed.

The code may be found on GitHub and the gem on RubyGems. Also don’t forget to checkout the fOOrth support site on http://www.foorth.org/.

The big sticking point, now under study, is the issue of defining instance variables in a procedure. This will likely require changes to the method mapping of instance variables, assuming this change is actually made.

In the mean time, best regards;

Peter Camilleri (aka Squidly Jones)

One more bit please!

gem_download_sept_2016

 Hello All!

Well it has been some time since I began working in Ruby and creating little gems of code for my page in the the RubyGems.org repository. When I looked at my stats this morning I realised that I had reached a milestone of sorts. The total downloads counter showed a value of 65,881 downloads!

Now, it is a well known fact that a sixteen bit, unsigned integer can count up to 65,536. This means that my download total no longer fits into that data type. While this is not nearly as impressive as the video Gangnam Style blowing through the YouTube signed 32 bit integer view count limit of 2,147,483,647, it is my accomplishment! 😉

One more bit please! That’s right, I’d like a seventeen bit integer, Thank You Very Much!

Best regards;

Peter Camilleri (aka Squidly Jones)

Unconventional tools, #1

Hello all;

In this series of posts, we shall take a look at some unconventional tools that help solving various programming (and other) problems that often come up. For this first article, I want to tackle a problem that actually made the list of the two hardest problems in computers science. Here’s that list:

  1. Selecting meaningful names.
  2. Cache coherency and invalidation.
  3. One off errors.

Ignoring the one-off error in the list, the first issue is selecting meaningful names. Way back in the bad old days, it was considered acceptable to have a program full of entities like i, j, k, and pmz21. Variable names that were meaningless, supported by a mountain of comments that were often completely out of data and/or misleading.

Slowly, computers improved and space was no longer so violently cramping the writers style anymore. This however lead to the opposite problem of names like:

if (index_selection > the_number_of_selections_available_in_the_menu)
  display_a_pop_up_error_mesage("Oops");

Names so verbose that they obscure the intent of the code.

What we really need is names that are meaningful and not verbose. It boils down to a matter of selecting the right words. Regardless of what programming language I use, I program in the english language. There are over a million words defined for that language and I can assure you the I do NOT know them all. For those where english is not their native tongue, the problem is even worse.

What to do? Use a tool to help navigate the possible words for the job: Thesaurus.com!

Let’s try a real example I encountered just this morning. I was looking to update the description of my mini_readline ruby gem. I wanted to describe the fact that it came with four sample auto-completing thing-a-ma-bobs. The only word that came to mind was “engine” that sort of worked but sounded to mechanical. I wanted a word that would express the idea of work done on the user’s behalf. So of to the thesaurus web site and punched in engine… A large page of results including the word I sought for: agent!

So the next time you struggle with one of the most difficult tasks in programming (and writing too) consider using a free tool to ease the burden of picking that one in a million perfect word for the job!

Best regards;

Peter Camilleri (aka Squidly Jones)

Gem Download Study: The first 12 weeks.

As most of the readers of this blog know, I release a lot of Ruby language gems. They are announced on this web site regularly. Now it is possible for me to check how often these gems are downloaded. You can see that data here.

Now fairly early on, the view was expressed that most (if not all) of downloads I was seeing were the result of “bots” that grab everything they can find and gather the results for their industrial overlords, most likely in China. For some time I’ve wondered if that was true. To be honest, I hope it isn’t, but the nagging doubt never left me.

So I came up with a plan. I would track the downloads of my gems on a weekly basis for 12 weeks and see if I could see some trends that might indicate the nature of the download traffic. Well, it’s been 12 weeks and here are my initial results:

Gem_Downloads_12Now I know it’s a bit of an eye chart, but just click on it to get a better view of the data. Maybe even open it in another tab for easier reference.

The first thing that becomes visible is that the slopes of the lines, and thus the rate of downloads, is not the same for all of the gems. Some of the gems are clearly more popular than others.

Even more telling, is that for one gem, “vls”, a change was made recently, that described the use of this gem with the popular “rails” web framework/library/system. Once this appeal to this very active group was made, the rate clearly began an sharp upward trajectory.

Now this is only 12 weeks worth of data, so one should not read too much into these results, but they seem to indicate that at least some of these downloads can be traced to real people. Time will tell. I plan on my next study release at the 24 week point.

Yours Truly

Peter Camilleri (aka Squidly Jones)

Announcing the Mystikó Encryption gem

Let me start by saying that there is some very important advice to any programmers who might be tempted to create the own encryption security systems. That advice is well summarized by the word: DON’T!

Most home-brewed encryption schemes, including many used in vital embedded systems, are laughably weak. Straying from proven, working standards is a recipe for disaster and loss. So I say again: DON’T!

Having said that, why am I now going to discuss the Mystikó encryption gem? Very simply, this gem is only intended for educational and amusement purposes. Under no circumstances should it be construed to be a serious encryption tool or fit for protecting actual property, privacy or security.

So why do it? Why not? It was a fun challenge and a good follow on for my earlier work with pseudo-random number generators. It was also an interesting way to learn a little bit more about this fascinating topic while having fun with Ruby programming.

The gem itself is at: The Mystiko gem. The source code may be found at: Source Code. The source code also comes with a special added treat! In the Challenge folder of the source code you will find the file “challenge.bin” This file contains a paragraph of famous text encrypted by the Mystiko gem.

At this time, I am still working on raising some funds/prizes for the first person to crack this code, after which the plain text and the key will also be published so that others may verify the winning entry!

So, what shall happen? Will a jury rigged encryption program stand the test of time? or will it be cracked soon as predicted by common wisdom? Time will tell!

Yours Truly

Peter Camilleri (aka Squidly Jones)

Announcing the vls utility.

One of the joys of modern programming is the ability to utilize external code libraries to speed up development, saving time, and reducing wasted effort. The Ruby language is especially blessed in this regard with its system of code gems. Instead of the narrow minded NIH (not invented here) mindset, the world has become our tool chest.

There is, however, a downside to this modular nirvana. Versioning! To be precise: Am I using the correct version of each (and every) little code gem? This problem goes back all the way to the old Visual Basic days. Back then it was called DLL Hell, as developers struggled to maintain a myriad of cryptic, often poorly documented binary files.

Now Ruby does have the bundler utility that allows gem versions to be specified, but what if you simply want to know: When I use this application, what modules/classes are being used?

The vls utility answers that question. To use this simply enter:

$ vls <names>

where names are a list of gems/files to be required before the modules are listed. Here, see an example of this in action:

$ vls fOOrth
vls (VersionLS): 0.1.0

Bignum, 0.0.5
Complex, 0.0.5
Date::Infinity, 0.0.5
FalseClass, 0.0.5
Fixnum, 0.0.5
Float, 0.0.5
FormatEngine, 0.7.2
FullClone, 0.0.5
Gem, 2.2.2
InArray, 0.1.5
Integer, 0.0.5
MiniReadline, 0.4.8
NilClass, 0.0.5
Numeric, 0.0.5
Rational, 0.0.5
Regexp, 0.0.5
RubySscanf, 0.2.1
SafeClone, 0.0.3
Symbol, 0.0.5
TrueClass, 0.0.5
XfOOrth, 0.6.1

The vls gem may be found at: https://rubygems.org/gems/vls and the source code lives at: https://github.com/PeterCamilleri/vls.

Yours Truly

Peter Camilleri (aka Squidly Jones)

The Durham Makerspace

I’ve written about meetup groups in the blog space before and I continue to enjoy them as a social and educational outlet. Yesterday, I attended my first East-End Tech Meetup. I must say that having a group closer to home is a lot nicer than a 3 hour round trip into downtown Toronto. The real excitement, for me however, was the topic. Here it is from the meetup description:

A remarkable young member of our community is planning to launch a GTA-class Makerspace right here in Oshawa. He wants to present his ideas to our community, elicit feedback, generate more ideas & strategies, as well as gather the much needed support from our citizens here in the Durham Region. This is no small undertaking and it will take the effort of many to back his execution.

This would be a substantial asset to Oshawa and its people so spread the word. Tell your friends & neighbors. Every bit of support is needed to make (pun) this a reality.

Now, I’ve visited maker labs before. They were all far away, very expensive, and difficult to join, especially for an older person such as myself. Let’s just say that I did not feel welcome.

This group is different. I suppose it is at least partly the thrill of getting in on the ground floor, but there was an energy present that I had not seen or felt before. There was also an open exchange of ideas and many diverse contributions were being sought after. My impressions were very positive.

The effort is still in its infancy, but they have a facebook page. Take a look and hopefully find a resource to express your creativity through engineering and making.

Yours Truly

Peter Camilleri (aka Squidly Jones)

The Clone’s Family Tree

The Ruby programming language has two methods for duplicating data. These are “dup” and “clone”. While these methods are quite useful, they both suffer from two shortcomings:

  1. In Ruby, if an attempt is made to clone (or dup) an immutable data item like a number, an error occurs. The justification for this uncharacteristic strictness is not at all clear, but it does mean that the clone (or dup) operation must be applied with great care.
  2. The copying process used by both clone and dup is said to be a shallow (or incomplete) copy. While the target data structure is copied, any internal data structures are not. References to those data remain aliased in the copy.

I started off to create a gem to resolve these issues. I ended up creating a family of  four gems that are tailored to the exacting data copying requirements of the application. Here is a summary of those gems:

Depth / Action Need to copy all data and metadata attributes? Need to copy data only?
Only need a shallow copy? Use the safe_clone gem. <Source> Use the safe_dup gem. <Source>
Need a full, recursive copy? Use the full_clone gem. <Source> Use the full_dup gem. <Source>

Notes:

  • Since none of these gems override the default clone and dup methods, the default behaviors remain available. Further, if multiple, differing requirements exists, more than one family member gem may be employed in the same project without fear of conflict.
  • If multiple family gems are employed, they will each need to be installed and required into the application. See each gem’s github source for details.
  • Meta-data attributes include the frozen status and singleton methods. However the tainted status is always copied.

I hope you find these little gems as useful as I have found them to be. I you like them, give the code repository a star as a sign of approval!

Yours Truly

Peter Camilleri (aka Squidly Jones)