Announcing the vls utility.

One of the joys of modern programming is the ability to utilize external code libraries to speed up development, saving time, and reducing wasted effort. The Ruby language is especially blessed in this regard with its system of code gems. Instead of the narrow minded NIH (not invented here) mindset, the world has become our tool chest.

There is, however, a downside to this modular nirvana. Versioning! To be precise: Am I using the correct version of each (and every) little code gem? This problem goes back all the way to the old Visual Basic days. Back then it was called DLL Hell, as developers struggled to maintain a myriad of cryptic, often poorly documented binary files.

Now Ruby does have the bundler utility that allows gem versions to be specified, but what if you simply want to know: When I use this application, what modules/classes are being used?

The vls utility answers that question. To use this simply enter:

$ vls <names>

where names are a list of gems/files to be required before the modules are listed. Here, see an example of this in action:

$ vls fOOrth
vls (VersionLS): 0.1.0

Bignum, 0.0.5
Complex, 0.0.5
Date::Infinity, 0.0.5
FalseClass, 0.0.5
Fixnum, 0.0.5
Float, 0.0.5
FormatEngine, 0.7.2
FullClone, 0.0.5
Gem, 2.2.2
InArray, 0.1.5
Integer, 0.0.5
MiniReadline, 0.4.8
NilClass, 0.0.5
Numeric, 0.0.5
Rational, 0.0.5
Regexp, 0.0.5
RubySscanf, 0.2.1
SafeClone, 0.0.3
Symbol, 0.0.5
TrueClass, 0.0.5
XfOOrth, 0.6.1

The vls gem may be found at: https://rubygems.org/gems/vls and the source code lives at: https://github.com/PeterCamilleri/vls.

Yours Truly

Peter Camilleri (aka Squidly Jones)

The Durham Makerspace

I’ve written about meetup groups in the blog space before and I continue to enjoy them as a social and educational outlet. Yesterday, I attended my first East-End Tech Meetup. I must say that having a group closer to home is a lot nicer than a 3 hour round trip into downtown Toronto. The real excitement, for me however, was the topic. Here it is from the meetup description:

A remarkable young member of our community is planning to launch a GTA-class Makerspace right here in Oshawa. He wants to present his ideas to our community, elicit feedback, generate more ideas & strategies, as well as gather the much needed support from our citizens here in the Durham Region. This is no small undertaking and it will take the effort of many to back his execution.

This would be a substantial asset to Oshawa and its people so spread the word. Tell your friends & neighbors. Every bit of support is needed to make (pun) this a reality.

Now, I’ve visited maker labs before. They were all far away, very expensive, and difficult to join, especially for an older person such as myself. Let’s just say that I did not feel welcome.

This group is different. I suppose it is at least partly the thrill of getting in on the ground floor, but there was an energy present that I had not seen or felt before. There was also an open exchange of ideas and many diverse contributions were being sought after. My impressions were very positive.

The effort is still in its infancy, but they have a facebook page. Take a look and hopefully find a resource to express your creativity through engineering and making.

Yours Truly

Peter Camilleri (aka Squidly Jones)

The Clone’s Family Tree

The Ruby programming language has two methods for duplicating data. These are “dup” and “clone”. While these methods are quite useful, they both suffer from two shortcomings:

  1. In Ruby, if an attempt is made to clone (or dup) an immutable data item like a number, an error occurs. The justification for this uncharacteristic strictness is not at all clear, but it does mean that the clone (or dup) operation must be applied with great care.
  2. The copying process used by both clone and dup is said to be a shallow (or incomplete) copy. While the target data structure is copied, any internal data structures are not. References to those data remain aliased in the copy.

I started off to create a gem to resolve these issues. I ended up creating a family of  four gems that are tailored to the exacting data copying requirements of the application. Here is a summary of those gems:

Depth / Action Need to copy all data and metadata attributes? Need to copy data only?
Only need a shallow copy? Use the safe_clone gem. <Source> Use the safe_dup gem. <Source>
Need a full, recursive copy? Use the full_clone gem. <Source> Use the full_dup gem. <Source>

Notes:

  • Since none of these gems override the default clone and dup methods, the default behaviors remain available. Further, if multiple, differing requirements exists, more than one family member gem may be employed in the same project without fear of conflict.
  • If multiple family gems are employed, they will each need to be installed and required into the application. See each gem’s github source for details.
  • Meta-data attributes include the frozen status and singleton methods. However the tainted status is always copied.

I hope you find these little gems as useful as I have found them to be. I you like them, give the code repository a star as a sign of approval!

Yours Truly

Peter Camilleri (aka Squidly Jones)

Hidden Traps: String Mutations

In the world of Sci-Fi movies and video games, mutants are not always a bad thing. In fact, sometimes, the mutants are the hero or the key to the hero’s success. On the other hand, no matter how good their intentions, trouble always seems to follow them.

In the world of programming, data can be mutated as well and it attracts trouble there too. Data is mutated when its internal state is modified. The alternative is data that is immutable. This means that the internal value cannot be modified, it can only be replaced with a new value.

Let’s see an example of immutable data in action first:

a=42
b=a
a+= 4
puts a   #Displays 46
puts b   #Displays 42

This is the safe, sane case. Changes to the variable a do not affect the variable b. Now, in Ruby, strings are mutable. Let’s see how this plays out.

a=”hello”
b=a
a << ” world”
puts a    #Displays: hello world
puts b    #Also displays: hello world

In this case, the string data in a and b was mutated. The change to a was expected. The change to the seemingly unrelated variable b could be a nasty surprise and a source of bugs. I can say that this very issue is without doubt the number one source of trouble in my own code.

The sensible question to ask is: Why are string mutated at all? Why not always protect the programmer from this issue? The answer is performance. Creating new instance of strings all the time is wasteful of resources and places a heavy load on the memory management system. Let’s see how this issue is handled by two leading languages: C# and Ruby.

C#

In C#, the mutation of strings issue is handled by giving the programmer choice. The default, goto class for strings, namely the String class, is fully safe and immutable. Thus for simple programming tasks and less experienced programmers, the obvious choice also is the easiest to get right. Maybe not super-fast, but the expected results.

When (and where) performance issues do arise, the StringBuffer class exists that can be more efficient because is allows data mutation. In fact, it embraces it with special mutating methods, not part of the String class. Since a conscious choice must be made to allow and use mutation, hopefully, the programmer will focus there attention on those areas, thus avoiding nasty surprises.

Ruby

In contrast, Ruby (currently) allows string mutation by default. This means that any string at any time could be mutated. This causes a lot of problems. There are some possible answers:

  • When assigning a string, clone it. A statement like: a = b.clone breaks the connection between a and b by making a full copy of the string. It works, but this can be slow.
  • When creating a string, freeze it. This looks like: b = “Hello”.freeze and now any mutation will raise an error that reveals the bug at the point it first occurs. This is not slow, but requires thorough testing to avoid exceptions in the field. You also need to remember to use the freeze in the first place.
  • In upcoming versions of Ruby (version 3 I believe), strings will be frozen by default. You will now need special syntax to create mutable strings. This will look (probably) something like b = String.new(“Hello”). This suffers from the fact that it is likely to break a lot of older code.

These two languages show two divergent approaches to the issue of string mutation. Now I really like Ruby, and C# is a pretty decent language too. On this issue though, C# looks like the clear winner here. Safe by default, gentle on programmers, efficient where needed, and above all, not breaking older code. To me it’s a slam-dunk!

What are your thoughts? Does another language do an even better job? I’d love to hear from you, so chime in with a comment, idea, or objection!

Yours Truly

Peter Camilleri (aka Squidly Jones)

Article Update: April 15, 2016

I am pleased to announce the release of the fOOrth 0.6.0 language system. The major change in this version is to split the String class into an immutable String class and a mutable StringBuffer class. Even though fOOrth is based on Ruby, the underlying architecture is more than flexible enough to accommodate this change.

fOOrth

fOOrth_logoIt is with great pleasure that I am finally able to announce the release the initial beta of the fOOrth programming language system version 0.5.0. This language, written entirely in pure Ruby is released both as a Ruby gem and in Source Code on github.

So, what exactly is the fOOrth programming language system anyway? The source code repository has this to say about the project:

The fOOrth language is an experimental variant of FORTH that attempts to incorporate object oriented and functional programming concepts. It also tries to extrapolate an alternate reality where FORTH was not frozen in the past, but continued to grow and develop with the times. Above all this project is the result of nearly 30 years of thought on the design of threaded compilers and languages with simplified grammars and syntax.

Also included in the docs folder are a User’s Guide and Reference in both open office and PDF formats.

A lot of work and effort has gone into this personal labour of love. For example, I have discovered that writing good documentation is a lot harder than it looks. It is certainly harder than just writing code. I have also found however that having to explain what is going on in plain, simple, and easy-to-understand terms has resulted in many improvements to the code that would not have happened otherwise. In effect, it served as a sober, detailed review of the code being generated.

This is still only a beta, and there is still a great deal more work to do. I hope you find it useful or at least interesting. If you like what you see, give the code a star to show your support. Suggestions, comments, complaints and ideas are always most welcomed.

Yours Truly

Peter Camilleri (aka Squidly Jones)

The redesigned minitest_visible gem (v0.1.0)

Just because code works, doesn’t mean it’s the correct code. A case in point is my own minitest_visible gem. This little bit of Ruby code adds simple progress tracking to the testing process. In any programming, and especially in a  dynamic language like Ruby, testing is vital. So is having faith in those tests.

Now the standard minitest testing system is awesome! It is full of helpful methods for confirming the correctness of code. It is however a little terse regarding progress. That is where minitest_visible comes in. When used, it lets you know the version of minitest being used and prints out the name of each test file as it is processed.

A fairly simple task. Yet, when I came up with the first version of this code, it was clunky and ugly, but it worked. I suppose I had other matters pressing at the time, but I let things stand at “good enough”

Recently, I began to think that there must be a way to make the code better, cleaner, and easier to use. It seems that I have indeed learned some stuff in the last year because I am now writing about a new version that is far simpler and streamlined.

OK, let’s be clear: The old way of doing things is still supported, but prints out a message about the needed changes. Test still pass though so disruption and panic should be minimal.

The new minitest_visible can be found https://rubygems.org/gems/minitest_visible and the source code is at https://github.com/PeterCamilleri/minitest_visible.

Yours Truly

Peter Camilleri (aka Squidly Jones)

A Double Bill!

Having just announced format_engine version 0.4.0, things seem to be speeding along as I now announce format_engine version 0.5.1 and ruby_sscanf version 0.1.1.

These two gems are rather tightly coupled. The ruby_sscanf gem uses the capabilities of the format_engine to create a Ruby translation of the POSIX ‘C’ sscanf routine. This allows a formatted string to be processed into the data elements contained within. The new version of the format_engine is required to add some features required by the ruby_sscanf gem.

I thought a great deal about the idea of merging the ruby_sscanf gem into the format_engine, but in the end, concluded that they did different things despite the high degree of coupling. Further, ruby_sscanf modifies (or monkey patches) the base String class, and that was something that might not always be desired. So two separate gems.

These may be found at:

format_engine gem, format_engine source code

ruby_sscanf gem, ruby_sscanf source code 

I hope all Ruby programmers find these programming tools to be useful.

Yours Truly

Peter Camilleri (aka Squidly Jones)

Format Engine Version 0.4.0

I am pleased to announce the new version of the format_engine gem. This blog has been highly remiss in keeping up with the releases of new code. So, we are skipping ahead from version 0.0.2 to 0.4.0. After a hiatus of six months, the new version has improvements, mostly to the parser side of the engine.

  1. Parsed spaces may match zero or more input spaces so that it is no longer necessary to predict the exact number of spaces between data elements.
  2. The parser now supports strings with regular expression sub-sets like %[abc] which matches the input in the set “a”, “b”, and “c”. Also supported is exclusion sub-sets like %[^abc] which matches the input not in the set “a”, “b”, and “c”.
  3. Improved documentation in the README.md file.

The gem may be found at Ruby Gems and the source code at GitHub.

Yours Truly

Peter Camilleri (aka Squidly Jones)

Hidden Traps: Floating Point Math.

As a life long programmer, I’ve programmed a lot of calculations over the years. Yet, I was recently astounded to hear expressed the opinion that “Computer computations are error free and flawless”. The reasons that this is not (and never will be) the case are fairly deep and fundamental.

Mathematics routinely deals with matters of the infinite like the infinite digits in the value of the famous irrational number π (called pi). Engineers and computers on the others hand live in a world of the finite: finite memory, finite processing power, and finite time. The IEEE standards for computer arithmetic are marvels of ingenuity. They are also a compromise between the needs of speed, storage, and accuracy.

Let’s see a real example and test the associative property of addition. This crucial, fundamental rule of math says that:

(a+b)+c = a+(b+c)

Now lets try to program this. We’ll use Ruby for brevity, but the results would be the same for virtually any programming language that uses the computer’s floating point hardware.

a = 1.0e20
b = -1.0e20
c = 1.0

puts “(a+b)+c = #{(a+b)+c}”
puts “a+(b+c) = #{a+(b+c)}”

The output of this little snippet of code is:

(a+b)+c = 1.0
a+(b+c) = 0.0

The two results should be the same but they’re not. In one case, the use of a finite amount of memory to represent the numbers has caused an important “bit” of data to fall off the “end” and be lost. Addition in computers is not always associative.

Are there ways around this? Sure, you can improve things. In Ruby I can switch from Floats to Rational data, and this problem mostly goes away. The new program will consume more memory and take longer to run. Eventually, if the required precision becomes excessive, the program will slow to a crawl and/or run out of memory. Processing power and memory remain finite no matter how sophisticated the tools we use.

The moral: Know your math, yes, but realize that computers can’t do math, they can only approximate it! So know the limitations of your tools too.

Yours Truly

Peter Camilleri (aka Squidly Jones)

Version 0.0.2 format_engine released.

In other news, a very minor bug fix for the format_engine gem (just recently released) is now available. This corrects an issue with the handling of invalid format strings. A very minor fix and most need not rush out a get the update. Still found on RubyGems and GitHub. The plus side is that as a minor bug fix, and now passing even more tests, this is a low risk upgrade.

As always, comments and suggestions are welcomed. If problems or questions are encountered, please raise an issue under the GitHub project. I do watch the issues very closely so will be sure to give any reports immediate attention.

Yours Truly

Peter Camilleri (aka Squidly Jones)