A little gem… [Updated]

[Update] HELP! 

Let me be clear on this point. I love programming in Ruby. I haven’t been as excited about a programming language since the time I got to work on a major project in Smalltalk. Ruby is awesome! I am having a LOT LESS fun with RubyGems.org. I have been muddling my way through reams of confusing documentation, knocking down problem after problem.

Now one still remains. My code contains rdoc documentation. I chose rdoc as it is the system used in the “pick-axe” book. Issue 1a) When I test install my gem locally, the rdoc docs are not built unless I explicitly add the deprecated –rdoc option. 1b) When I push my gem to RubyGems.org, the documents link points to an empty page with no info. It’s so bad, I’ve taken to adding this note to the description:

Please see the homepage for docs.

Can anyone tell me what subtle nuance I’ve gotten wrong here? I am at my wits end!

[End Update]

A very brief announcement. I just released my first ruby gem. It’s called option_list and it is a ruby class for processing symbolic function arguments. It makes it easier to process arguments that are clearly documented and understandable. So instead of:

my_fun(">", false, true, 66)

You could use:

my_fun(:nohistory, :paginate, prompt: ">", page_len: 66)

Long winded? Maybe. Clearer? I think so! Now I know; this is NOT going to set the world on fire. In fact it’s more of a “Hello Ruby World” than anything else, but please take a look.

The documentation for it can be found here and the gem itself is found here!

Any feedback on this gem or other topics would be most appreciated.

Thanks, Peter Camilleri (aka Squidly Jones)

PS: While my code contains full rdoc info, the documents link on RubyGem.org goes to an empty page for some reason. If someone could explain my error, I would be most appreciative and grateful for the help..

Impressions of Masters 2012 [Updated]

Welcome to Masters!

First a disclaimer. There is no way that the Microchip 2012 Masters Conference can be summarized here. It is simply far to vast in scale and extent to do it justice. Instead, consider this to be a series of first person impressions, notes, recollections, and anecdotes.

To put things in perspective, this was my seventh time attending the Masters conference. Previous years attended were 2000, 2001, 2002, 2004, 2009, and 2011.

.

The front door!

I’m not quite sure how long it’s been there, but this year the conference was held at the luxurious J.W. Marriott. Here’s the grand front resort entrance. It goes without saying that the hotel, its rooms, conference halls and public spaces where all luxurious and well appointed. The conference included three sumptuous meals a day with beverages, fruit and “treats” for those who got the munchies in between study courses.

.

The “swag” of the year!

As is common at such events, all attendees went home with a little “swag” to remember their participation. In past years, little technology demos were often given out. The last few years this has been replaced by a note book, polo shirt and course notes thumb drive combo. This is a little disappointing as the gadgets were always fun, but to be honest, other than some fun playing with them, not much came of them, so the cost savings is sort of understandable, even if it is a bit of a downer.

As said earlier, this is my seventh Masters. Each year I have the same problem. There are many more courses that I would like to take than there are time slots in which to take them. This year there were 36 courses offered covering a wide range of topics. That’s too many to list so instead here is the Masters 2012 Event Guide. The class list is on pages 11 through 31. As for myself, I was kept busy learning about:

  • Upcoming developments in Microcontrollers and Development Tools.
  • Learning more details of the FreeRTOS software. I was fortunate enough to win the programming challenge and got a copy of the book Microchip PIC32 Edition of the FreeRTOS user’s manual. I’ll be doing a review of that soon.
  • Using the Microchip Graphics Display Designer (GDD-X)
  • Robust I2C communications with the PIC I2C peripherals.
  • USB: Both the Generic Driver and CDC class devices.
  • Bootloading, Application Mapping and Loading Techniques on a PIC32.
  • An examination of Advanced Touch Screen Technologies: Projected Capacitance Touch.

As if that were not enough, the time slot between the end of studies and dinner was also utilized each evening.

The Keynote Address (Picture from Facebook)

 On Wednesday there was the interesting (OK not THAT interesting, but were did I have to go?) keynote address by Steve Sanghi, the CEO of Microchip. Mr Sanghi has a direct, down-to-earth approach that is most refreshing and while not edge of your seat stuff, it was still very informative to hear the corporate perspective.

On Thursday, I was delighted to see the return of Don McMillon! Don is the self-proclaimed “Funniest Engineer in the World!”. Even though this was his second time at masters, the show was mostly new material and I had a great time. With all the serious engineering work going on, some levity was welcomed.

And then there’s Friday… On Friday, there was a Casino Night. During this event, I made the earth shattering discovery that I truly suck at poker. Thank goodness the $1,000 I lost was only play money (Unlike some 2012 USA Presidential candidates, I do NOT make casual $10,000 bets!).

Finally, I have left what is possibly the best part for last. After dinner each evening, there were semi-formal discussion groups with various microchip teams in attendance to answer questions. I attended the XC16 and XC32 discussions. I learned a great deal of useful info. Here’s a sampling of some tasty tid-bits:

  • XC32++ is ready! That’s right C++, with a FULL library by Dinkumware, a highly respected library specialist vendor. XC16++ is still in the very early phases and is rather much much further off.
  • New PIC32s are coming that will support new modes that are faster and more compact than the current MIPS16 mode. Support is coming for higher speeds and much larger memory and a external memory interface. However, no target dates were given. Sigh… Marketing!
  • One surprise was getting more clarity on the Errata Issues that I have written about before. It now seems that the first issue (with the prefetch cache) has only ever been seen with highly unusual configurations, not used by Microchip tools. Most developers need not worry about it. The second issue (with double writes to I/O registers) is more widespread and needs to be taken seriously. In the mean time, new silicon is coming out very soon (weeks?) that address these issues. We should be seeing an Errata Update in a little while and I will post it here too.
  • As the session was closing, somehow the discussion went to the topic of the use of assembly language by C programmers. To make a long story short, I indicated that the ONLY time I have ever used assembly language was to provide the special interrupt prologue and epilogue required by the FreeRTOS to save memory. At this point it was suggested that this was better done by the compiler itself and the Richard Barry (of FreeRTOS fame) should be contacted to see how progress could be made on this front. We’ll see what happens, but that would be a cool addition to XC32.

There was one important thing that was missing though: There was no break out session for the XC8 compiler group. Perhaps they were too busy?

It seems that Santa works for Fedex!

Oh and I forget… The really final final, even better thing about all the masters conferences is the deep discounting of developers tools. I felt like a kid in a candy shop with all those wonderful toys to contemplate! I admit that I had to restrain myself to a few choice selections. I’m sure I will be writing about them more very soon.

Perhaps I should start doing a mail bag segment on a You-Tube channel where I open my (interesting) mail? Then again, perhaps not.

.

As always, your thoughts, comments, suggestions and encouragement are most welcome!

Peter Camilleri (aka Squidly Jones)

PS: I was giving some thought to the idea of FreeRTOS prologue/epilogue support in XC32, and it seems that it need not be complex.One idea would be to support the idea of an “Interrupt Stack” which would be used by ISRs. This option would support FreeRTOS and provide benefit to any case where multiple stacks are in use. This would be an easily set up option for compilers and the linker could allocate the space in a manner similar to the way the heap is currently allocated.

[Update] Shortly after the conference, I exchanged emails with Joe Drzewiecki of Microchip regarding this idea. This correspondence follows:

Hello Joe;

I want to thank you again for a wonderful Masters Conference and especially for the very informative break out session on the XC32 compiler. The discussions covered a lot of ground and I learned a lot of new stuff.

I was thinking about the prologue and epilogue code for ISRs in FreeRTOS, and it occurs to me that what is involved here is the use of a special “Interrupt Stack” area. A stack set aside for use by ISRs. Implementation of this concept would be of use beyond FreeRTOS and include any system that runs multiple stacks.

I have updated my web site with a review of Masters 2012 here: http://teuthida-technologies.com/ and included a bit about Interrupt Stack support in XC32.

 I look forward to future developments;

 Best regards; Peter Camilleri

 and here was Joe’s reply:

Hi Peter!

Thanks for the props and the suggestions. We’ve had a very spirited discussion about your prologue/epilogue suggestion (go figure! :).

We’ll continue to discuss this, but may not act on it right away. There are many subtleties to it.

 Hope to see you next MASTERs,

Joe

Thanks Joe; It’s good to know I can still stir things up! 😉

Squidly’s Progress

I am glad to say that I have made some major progress in one of my MPLAB-X errata issues. Specifically, the issue of the debugger being unable to resolve the scope of local variables and parameters. After concluding that the support forums had no answers, I bit the bullet and created a support ticket:

In MPLAB-X debugger, function parameters display as “Out of Scope” when they are most clearly NOT out of scope. This forces all sorts of copying non-sense that is very multi-threading risky.

To make a very long story mercifully short, 10 Microchip support responses and 7 customer updates later we finally reached the following point:

Hi Peter,

After further investigation, it is observed that it is a MPLABX issue and not a[n] optimization issue. This will be corrected in the future version of the MPLABX IDE. The concerned team is working on it.

I hope it helps. If you need further support/clarification on the same issue, please update this ticket.

Regards, Microchip Technical Support

Take it from me, I’ve worked in code maintenance and I know that sometimes recreating a problem seen by the customer can be the most difficult part of the process. Getting a firm grip on the nature of the issue is essential to making any progress in resolving it. We are finally over that enormous hurdle and I am confident that progress will be made on this matter!

Finally; On a somewhat less serious note: I am very pleased to announce that I have passed the milestone of 100 postings in the Microchip Technical Support forums. Hooray for me 😉 Hopefully, my postings have been of value or helpful or at the very least, not harmful or incorrect!

As always, your comments and observations are welcomed!

Peter Camilleri (aka Squidly Jones)

C32 Impressions (3/23/2012)

In my PIC32 programming I have three primary development tools: The MPLAB-X IDE, the MPLAB ICD-3 programmer/debugger, and the C32 V2.02 “C” compiler. In this post we are finally going to take a look at the compiler. Like many compiler projects, C32 is a subset of the open source GCC 4.5.1 compiler and the non-proprietary bits are available in source format. Some major features absent from C32 include: C++ support, 128 bit integers, decimal floating types, 16 bit float types, and fixed point (DSP oriented) types. Still, many useful extensions to “C” are included and the user would do well to download the GCC  4.5.3 documentation (the closest to 4.5.1 that I could find). I especially like the binary constants feature; for example: i = 0b101010;

C32 is available in essentially two major versions: A free, 60 day trial and Full. The Free version is a free download from the we site and has the the following restrictions specified in the compiler release notes:

“Microchip provides a free Standard Evaluation edition of the MPLAB C Compiler for PIC32 MCUs. The standard evaluation edition of the compiler provides additional functionality for 60 days. After the evaluation period has expired, the compiler becomes reverts to the lite edition, and optimization features associated with levels -O2, -O3, and -Os are disabled. In addition, MIPS16 code generation is disabled. The compiler continues to accept the -O1 and -O0 optimization levels indefinitely.”

The full version is available for purchase from Microchip Direct for a list price of $895. This price is routinely discounted for attendees of the annual Master’s Conference, but even so it it is still a rather large chunk of change. To gain a better understanding of compiler performance, I undertook to perform a series of benchmarks using C32 V1.11, V1.12, and V2.02 at the four levels of optimization measuring performance in space (size in bytes) and time (execution in seconds). To test with I created four programs. Here are my findings for each program:

Test #1 Circles

Draw 100,000 random circles on a graphical display (no H/W acceleration)

Test #2 Glyphs

Draw 200,000 (8 by 13) Character Glyphs on a graphical display (no H/W acceleration)

      


Test #3 Prime Number Sieves

Find all prime numbers less than 20,000 and do this 5000 times.

Test #4 Spirographic Patterns

Draw 200 repetitions of a complex polar plot diagrams reminiscent of the old Spirograph toy.

Benchmarks Summary

For almost all debugging, it is advised to use O(0) or non-optimized code. The code re-arranging done by the optimizer makes debugging very difficult. At least, optimization should be turned off for the source file being debugged. As for production code, there does not seem to be a great deal of difference in optimization setting 1, 2, and 3. Setting  “s” produces consistently smaller and slower code. Thus the main advantage of the full compiler versus the free one seems to lie mainly in that it can produce more compact code.

MIP16 Mode

There are options to force the compiler to use MIPS16 mode throughout a project, however given that most projects contain various routines not allowed to 16 bit, this approach will seldom work well. I prefer to use the attribute ability to mark individual functions as 16 bit. This gives me fairly fine grained control over where I favor space over time. For brevity I define:

#define mCode16 __attribute__((mips16)) __attribute__((noinline))
#define mCode32 __attribute__((nomips16))

Note that the mCode16 macro selects mips16 mode and the noinline option. The noinline is required as a work around to a minor compiler bug, but the avoidance of inline code makes sense if you are trying to save space. A simple example where space savings may be desired is:

void mCode16 emQryTouchRaw(emPTOUCHRAW rd)
{
    rd->x = xf;
    rd->y = yf;
}

Even in this trivial case, MIPS 16 mode saved 16 bytes of program space. This option is NOT available in the free compiler after the 60 day trial period expires.

Conclusions

For the most part, the C32 compiler is a like a good car engine. It does a good job and you don’t have to waste time fussing over it. Like any good tool, it lets you focus on the job at hand rather than constantly tweaking things. I run the FULL version and I am glad to have the flexibility it offers. Nevertheless, a great many projects can be accommodated by the powerful free version of C32, and tool cost should not be an impediment to trying out new ideas.

Command Line Options

The compiler command line includes many many options, most of which are not documented. They are listed in this PIC32 GCC Usage file for your convenience. Use at your own risk!

Peter Camilleri (aka Squidly Jones)

PS: While Microchip is proceeding with caution, there are indications that they may yet unleash the C++ capabilities inherited from GCC! And that too would be a good thing!

UPDATE (3/23/2012) On further reflection, I just remembered that I left out a major method of reducing code bloat that should work in all versions of C32. When creating code modules, it is not uncommon for some functions to not be called. Now, these could just be deleted, but that often means maintaining multiple versions of the code which can be messy. Two options, work together to automatically remove un-called code. In the picc32-gcc there is “Isolate each function is a section” and in the picc32-ld there is “Remove unused sections”. Together these options can result in substantial space savings with no loss of speed or debugging capability.

On another unrelated note, the image to the left is a screen shot of my spiro-graph test. The subtle color changes are on purpose. I added them so I could see the colors shift as I drew 200 copies of the pattern for my testing. Many many years ago, I was tasked with writing a CALCOMP pen plotter driver and scientific graphing package in FORTRAN. For my test data, I used spiro-graphic patterns back then too and I have been intrigued by them ever since.

MPLAB-X Update (3/15/2012)

Stop the presses! I may have spoken too soon. In my previous article MPLAB-X Impressions I wrote:

Earlier betas of MPLAB-X had severe problems with watch windows being unable to locate local variables, but those issues seem to have been sorted out in V1.00.

Well take a look of the following from a recent debugging session:

The green highlight indicates that the CPU is halted at the specified line of code. The message “Out of Scope” is NOT what you want to see when you hover over a local variable. It basically means that the debugger cannot locate the desired data and you are  on your own.

The work around for this very annoying bug is to copy the variables affected to statically scoped variables that CAN be resolved by the brain dead code. The little snippet in the picture above shows this strategy in action.

I must say that I am disappointed.  I really thought this annoyance was behind us! That said, V1.00 was still better than the beta, so progress is being made.

Peter Camilleri (aka Squidly Jones)

(3/15/2012) Further update: I have raised two tickets with Microchip concerning the troubles I have encountered. I can’t help but notice a lot of loud complaining on the support forums but not much action. If you see a problem, create a ticket to complain to the the folks responsible for fixing it! If you can’t be bothered to take the effort to do that, you’ve got nobody to blame but yourself.

MPLAB-X Impressions

This post is a change of pace from the previous ones. We’re going to step back and take a look at the development environment that I use in my current work. I speak of the  Microchip MPLAB-X (V1.00) IDE. I will cover the compiler I currently use, C32 (V2.01) at another time. MPLAB-X is Microchip’s new flagship IDE for PIC development tasks of all sorts. I first learned of MPLAB-X about a year ago while it was still in early beta. I confess that Betaphobia (not to be confused with Cetaphobia) ruled and I stayed away from it. However, all was not lost; I got to use MPLAB-X at last years Master’s Conference where I have to admit I was most impressed. After the conference, I switched to it, in spite of the fact that it was still in beta.

So some basics first: It’s free! Just download V1.00 and install. Yes, it’s a dreaded V1.00, but more on that later. I have long since taken advantage of the extensive customization capability, so while my MPLAB-X looks like this, your’s won’t unless you share my need to see as many lines of source code at once as possible:

The MPLAB-X IDE as configured by me.

MPLAB-X is a full featured IDE designed to integrate with a wide variety of compilers and development tools for the full line of Microchip products. There is so much much ground to cover that this is not so much a review as my impressions. One really good bit of news is that MPLAB-X is based on the open Net Beans IDE. I look forward to being able to develop and make available my own special pluggins. As with most IDEs it comes with project management, source code editing, project building, and debugging capabilities. Nice extras are even though it supports popular source code control systems, it also supports revision tracking on the local machine. Right out of the “box” when started, MPLAB-X displays a most helpful start-up page:

The Start Up page. Not just a nuisance!

In the Learn and Discover Tab you find Getting Started  material, Tutorials, a number of helpful Wizards, and links to community and support. While there is a tendency to skip over this page, You shouldn’t as it contains valuable information. I like to at least glance at it before moving on to my code.

Once into the development environment it is pleasant and easy to use. The editor includes many of the conveniences of modern editors such as color syntax, symbol and error highlighting, auto look up and completion of symbol names, code re-factoring, code navigation, and text formatting. The editing environment is a joy to use and I won’t go back to the old MPLAB, however I did run into one glitch here though. I noticed that sometimes when I entered a symbol, the editor would add a redundant include directive to the code. Apparently, the editor had lost track of the fact that the file was already included elsewhere.

To fix this, I unchecked the “Auto Insert #include Directives for Completed Identifiers: option under Tools -> Options -> Editor Tab -> Language Selection -> C++. See the illustration for the view at the end of this rather laborious menu navigation challenge. Once unchecked, the phantom include directives ended, but I was again responsible for adding them myself, which suits me just fine.

.

A sample call tree from my Trivial File System module.

Also present in MPLAB-X are some superb features to enhance code understanding.The call tree graph is especially useful in studying code that you did not write. What does this function call and where do the calls go is usually a very tiresome exercise. With the call tree, it’s simple to get great looking graphs like the sample to the left. These graphs also make great documentation for internal system specifications or for impressing peers, managers and pets. Well maybe not pets, they’re much more difficult to impress.

A major area where MPLAB-X shines is that of build management. The old MPLAB had only one build configuration per project.

Under MPLAB-X a single project can command multiple build configurations with different source files selected, preprocessor symbols defined and different compile and link options. All of this being very customizable in this easy to use dialog shown to the left:

In addition, if you select a file in the project tab, and right click for the menu and select properties, you are given a dialog that allows you exclude that file from the build or use build options different from the other files.

This is shown to the left. There is one fly in the ointment here. If you have set a great many paths and other options up, you will have to redo all of that effort for the exception file. It would be nice and save a lot of time, if the exception file started off with the same options it would have had as a regular file.

The final major area I want to touch upon is the debugger. MPLAB-X comes with an integrated debugger with all the usual features we’ve come to expect: The call stack, watch windows, breakpoints, memory and register windows, plus single stepping of all kinds and flavors. Now my experiences are based on my use of MPLAB-X with my ICD-3, and I must say that this is the weakest link of the MPLAB-X package.

For starters, the ICD-3 drivers seems to get lost every once in a while. Once lost, the only answer is to use the little MPLAB-X driver switcher utility to bring the ICD-3 back as an MPLAB-X device. This also happens if you are debugging and walk away in debug mode for a long while. On return the debugger will not exit. Again to recover, exit MPLAB-X and run the driver switcher to correct this. I suspect that Windows 7 maybe interfering and reverting to old drivers because it feels like it, but I can’t say one way other the other. Anyway you look at it, this does not auger well for long debug testing.

Earlier betas of MPLAB-X had severe problems with watch windows being unable to locate local variables, but those issues seem to have been sorted out in V1.00. A constant problem with advanced processors such as the PIC32 is the need to remember to turn off the optimizer for debugging. The optimizer completely scrambles the relationship between source code lines and machine language instructions to the point that single stepping seems to jump around the code randomly. This can be a problem if code space is tight, because unoptimized code is much bulkier. The answer is to compile most of the files optimized with only the file under test unoptimized.

I realize that my look at MPLAB-X has been far too brief to give any idea of the real flavor of the tool in use. I can say in summary that I am very glad I decided to make the leap of faith and give it a try. Even the little problems I encountered were opportunities to learn and improve my skills and for a V1.00 it’s pretty nice today. So my advice is to plunk down the $0.00 (after getting budget approval of course) and give MPLAB-X a try. I’m pretty sure you’ll like it!

Peter Camilleri (aka Squidly Jones)