Friday, July 26, 2013

Boost.Build (Part 3)

This entry is about how to link in an external library using bjam AND a little start into using boost asynchronous IO. I'm not going to cover asio programs because I feel that the blog in the references already covers this extremely well. The entries I'll be making aim to tackle troubleshooting and filling in information where it appears to be missing.

The tutorial in [1] breaks the programs down really well. However there were no instructions on how to build the programs! You won't find a libboost_asio* on your system because that's not what it relies on. The command I used to compile the example using g++ 4.7 was:

g++ main.cpp -L/usr/boost -lboost_system -pthread

That was the absolute minimum to get the test program to build. Please see the developer's blog in [1] for the small example- it's the equivalent of a "Hello World".

Using bjam it's as simple as:

lib boostsystem
    :
    : <file>/usr/lib/libboost_system.so.1.46.1
    ;

exe sampleProgram : main.cpp .//boostsytem :
   <threading>multi ;

I'm still a little confused about the syntax of a Jamroot, so I looked it up [2]. To break things down line by line:

lib boostsystem

This line specifies that we have a library that we are describing "lib" (this is known as a rule-name) and its identifier is "boostsystem". The identifier could have been anything but since I'm linking together an ASIO program boostsystem is very descriptive.

:

The items after the first colon in a specification are the "sources" or input files. Since an existing library doesn't really have source inputs its contents will be empty.

: <file>/usr/lib/libboost_system.so.1.46.1

The contents after the second colon are the requirements. I'm not sure what the file tag does in this case- but it probably reads that the file must exist in order for the specification to be met.

;

Simply the termination of the specification.

exe sampleProgram : main.cpp .//boostsystem :

This line specifies that we are describing an executable program "exe" and that it is called "sampleProgram". Notice that the first colon is on the same line and that the first area specifies inputs as main.cpp (the source file) and .//boostsystem (which is the library we specified first!). The sources are terminated with a colon.

<threading>multi ;

Remember how in the manual command line execution I specified -pthread? Adding this argument to the build does that for you. This is a cool way to do it especially if you are working across different platforms where the threading platform differs (Windows vs. Linux vs. BSD's). Simply invoke bjam in the same directory and you're all done! Your first ASIO program is now built.


REFERENCES

[1] - http://www.gamedev.net/blog/950/entry-2249317-a-guide-to-getting-started-with-boostasio/

[2] - http://docs.huihoo.com/boost/1-33-1/doc/html/bbv2/advanced/jamfiles.html

Thursday, July 25, 2013

Arduino Tools (avr-g++)

Introduction 

I'm becoming more and more curious about Arduino. I bought a board a long time ago and it is currently being used as a dust-collection mechanism. The first time it was powered on, I was disappointed that the only thing I could program in appeared to be watered down C (and my passion is C++). It's been a couple of years and in that time some research was done. As it turns out, there is a compiler to build Arduino applications!

Obtaining avr-g++ 

The first step was to get the compiler and toolchain. This worked rather nicely on Ubuntu:

> sudo apt-get install arduino

I came back to the command line and typed:

> avr-

And pressed Tab for completion. As it turns out, the installation of the Arduino module installed the avr-g++ toolchain. That g++ at the end makes me feel warm and fuzzy, kind of like when I see a terminal on the desktop with vim. Now we're getting somewhere! 

Ubuntu's package management system does something kind of funny with what it pulls down. The header files for the Arduino are located in: /usr/lib/avr/include/avr. That's unusual; why would header files be found in /usr/lib? That's besides the point. The first example [1] of a program written for avr used a header file called "wiring.h" - there is no wiring.h in the files downloaded. Keep in mind that the project shown in that example was built using eclipse. I'm going straight command line so this will not do. 

Obtaining Makefile's for Arduino Projects

I'm not the best one to talk to about Makefile's for Arduino. I can tell you that they will save you an enormous amount of time and frustration. As of this blog post, there are a number of makefile's out there to choose from. The one I'm using is "edam's Arduino makefile" [4]. It was not difficult to use, especially if you are already familiar with the Linux command line tool "make".

Obtaining and Using Arduino-specific Libraries

Getting the compiler is only half the battle. The other half is obtaining the Arduino-specific libraries that are useful when building "sketches". I'm including this as part of the article because it's good to know what their software is doing. Also the Makefile provided above also uses the Arduino-specific libraries. [5]

The Arduino library is separated into 5 different subdirectories: app, build, core, hardware, and libraries. Because this is a C++ blog, I won't trouble you with the silliness in any other directory but hardware- because that's where the C++ gold is. Navigate to:

Arduino/hardware/arduino/cores/arduino/

Here is the link on github where you can look at all of the source code for the library [6]. Looking specifically at Arduino.h, you can see it defines the stubs for setup() and loop() (around lines 117-118). If you look at the boilerplate code in main.cpp you'll see how it mostly comes together:

// Obviously this is going to include a lot of the hardware-
// specific internals, most of these will be macro'd out depending
// on the architecture of the board you selected (in my case ATmega328).
#include <Arduino.h>

int main(void)
{
    // Initialization of hardware.
    init();

   // USB Connectivity (if any).
#if defined(USBCON)
    USBDevice.attach();
#endif

    // Calling the stup() method we would write ourselves!
    setup();

    for(;;) {
        loop();
    }
}

That's the gist of the boilerplate code. When the Makefile runs, it will build your "sketch" code (C++ code that is only setup() and loop()) AND the boilerplate code and all of the code in this library. It's all linked together statically because we only deploy one hex file to the device at a time.

Hardware

After doing some research, I found out that the board I'm using is an ATmega328 [2]. This is useful because we need to know the architecture to pass into the avr-g++ compiler in order for it to enable the correct macros. The main header file to use is actually "avr/io.h".



When you plug your board in via USB there is a check you should do to make sure everything is connected and that is:

1) Connect the board via USB.
2) Check to make sure the board is powered by check the LED's on the board.
3) cd /dev
4) ls

Check the output for a device called ttyACM0. If that device exists you're good to go! 

Output Binaries

When you eventually do build the program for Arduino it should be a file with a .hex extension. This tripped me up originally because avr-g++ can produce a.out files if not given an output filename. The question you might have is, "How in the hell do I get this onto my device?" Great question. You want to use a program called avrdude. For my specific hardware here is the command I used to upload my hex file to the Arduino (USE THIS AT YOUR OWN RISK, I TAKE NO RESPONSIBILITY...EVER):

sudo avrdude -P /dev/ttyACM0 -c avrisp -p m328p -v -e -U flash:w:blink.hex -U lock:w:0x0F:m -F

The reason why this was executed with super user permissions was the access to the hardware. A more experienced linux professional could probably get this to work without the permissions but for the sake of this article I won't cover that. A lot of this I pulled from external sites so it's a little magical to me. The gist is that avrdude uses the device ttyACM0 and uses the programmer avrisp (-c avrisp) for the ATmega328 (-p m328p) and uploads my hex file (blink.hex) to the hardware. PLEASE NOTE: A couple of the sites I visited warned me about the force flag (-F) and why it shouldn't be used. I wouldn't do this unless you 1) don't care about your hardware, 2) understand fully what it is doing. I fall into the first group.


REFERENCES 



Custom New + Delete (Part 1)

The motivation behind wanting to learn this is:

1) Better understand how operator new works.
2) Build up towards custom allocation for fast pimpl's.
3) It's pretty cool.

For my first test of trying to understand how the custom new/delete worked, I implemented a small program. My preliminary search for custom new/delete didn't turn up a simple example, so here it is:


#include <iostream>
#include <string>

struct S
{
S() : value(13), message("Hello World!") { }
int value;
std::string message;

void* operator new(size_t i) 
{
std::cout << "CALLING CUSTOM NEW!" << std::endl;
return new char[i];
}

void operator delete (void* ptr)
{
std::cout << "CALLING CUSTOM DELETER!" << std::endl;
delete[] reinterpret_cast<char*>(ptr);
}
};


int main(int argc, char** argv)
{
    for(;;)
    {
S* s = new S();
std::cout << "s.value = " 
                  << s->value 
                  << ", s.message = " 
                  << s->message 
                  << std::endl;
delete s;
    }
}

Doesn't get much simpler. The points of understanding for this are:

1) operator new DOES NOT invoke the constructor! It merely allocates the memory that is later filled by the constructor. 

2) Always have a corresponding operator delete.

It's really that simple. I ran this program and watched the memory consumption using top. It didn't seem to budge so that made me feel good. 

Tuesday, July 23, 2013

Overloading (NOT) Partial Specialization

I haven't put up an entry the past few days because of the confusion I've had about overloading vs. partial specialization. There are a few things I'm absolutely sure of (because a book tells me so):


1) Templates can overload [1].

2) There is no such thing as partial specialization for a template function [1].


For point #1, it simply means that if you have a plain old function (no template parameters) and you also have a templatized function with the same name and template parameters, the templatized function can be used to overload. I like the example I found on the IBM page [2]. The example was:


template<class T> void f(T x, T y) { cout << "Template" << endl; }
void f(int w, int z) { cout << "Non-template" << endl; }
 
 
That is a nice simple example of how the function template overload works. We all know that this works and works really well.

Point #2 is where I had the most trouble. I pounded this statement into my head for the past four days; not only that this was also instilled in me from numerous C++ lectures. For me, the statement is better rewritten: "There is no such thing as partial specialation for a template function, there are only overloads." Think about it- you can overload functions with template parameters. Therefore the following is another example of simple template overloading:


template <typename T> 
void func(T t) { cout << "template <typename T> void func(T t)" << endl; }
 
template <typename T> 
void func(T* t) { cout << "template <typename T> void func(T* t)" << endl; }
 

These are both just overloaded functions. Two unique overloaded functions! Things were fine in my head until I saw the Dimov/Abrahams example (rewritten in terms of above):


// 1 
template <typename T> 
void func(T t) { cout << "template <typename T> void func(T t)" << endl; }
 
// 2
template <typename T> 
void func(T* t) { cout << "template <typename T> void func(T* t)" << endl; }
// 3
template <>
void func<int>(int* i) { cout << "template <int>(int* i)" << endl; }
 
// 4
template <>
void func<int*>(int* i) { cout << "template <int*>(int* i)" << endl; } 
int main(int argc, char**argv) 
{
    int value = 12345;
    func(&value);
}
 
Everyone knows when they see code like this and are asked, "Which overload is called?" that it's a trap. I would have said that the fourth would handle it- and I'd be wrong. The question becomes, why is #3 chosen and not #4? Moreover, say you have the following code (omitting #3:

// 1 
template <typename T> 
void func(T t) { cout << "template <typename T> void func(T t)" << endl; }
 
// 2
template <typename T> 
void func(T* t) { cout << "template <typename T> void func(T* t)" << endl; }

// 4
template <>
void func<int*>(int* i) { cout << "template <int*>(int* i)" << endl; } 
int main(int argc, char**argv) 
{
    int value = 12345;
    func(&value);
}
Surprisingly now, #2 is called! After studying for a bit, I came across what should be a bullet fact:

3) Overload resolution selects only a single, best-fit primary template.

Here is why function #4 is never called: it is an explicit specialization of function #1, NOT FUNCTION #2! Function #3 is a specialization of Function #2. Try the example above out and take out function #1:

// 2
template <typename T> 
void func(T* t) { cout << "template <typename T> void func(T* t)" << endl; }

// 3
template <>
void func<int>(int* i) { cout << "template <int>(int* i)" << endl; }
 
// 4
template <>
void func<int*>(int* i) { cout << "template <int*>(int* i)" << endl; } 
int main(int argc, char**argv) 
{
    int value = 12345;
    func(&value);
}

This WILL NOT COMPILE. The specialization for #4 requires that a primary template already exists. Function #3 can exist because its primary template is Function #2.  That is the key to understanding the Dimov/Abrahams problem, what specialization is tied to what primary template. Understand that first and the whole thing comes together.

Once it's understood the tie in between the primary and specialized templates we can now determine which primary template the compiler will choose to handle the overload. Given a choice between:


// 1 
template <typename T> 
void func(T t) { cout << "template <typename T> void func(T t)" << endl; }
 
// 2
template <typename T> 


void func(T* t) { cout << "template <typename T> void func(T* t)" << endl; }


and the input to func being an int*, the compiler will clearly pick option #2. The compiler then checks to see if option #2 has any specializations and if any of those fit. Since option #2's overloads were:

// 3
template <>
void func<int>(int* i) { cout << "template <int>(int* i)" << endl; }

It will pick #3 as the function used! Beautiful in how much sense it makes. If Option #3 isn't available, it would default back to the primary template which is Option #2. 

So in conclusion, the rules for understanding template function overloading are:

1) Templates can overload [1].

2) There is no such thing as partial specialization for a template function.

3) Overload resolution selects only a single, best-fit primary template.

4) Explicit specializations are chosen only if they fit the singly chosen best-fit primary template only, specializations applied to any other primary template are ignored.

 
REFERENCES

[1] - Exceptional C++ Style, Herb Sutter. Item #7: Why Not Specialize Function Templates?

[2] - http://publib.boulder.ibm.com/infocenter/comphelp/v7v91/index.jsp?topic=%2Fcom.ibm.vacpp7a.doc%2Flanguage%2Fref%2Fclrc16overl_fn_templates.htm


Tuesday, July 16, 2013

std::stringstream vs. scanf

Back in the olden days C-Programmers would use a facility called scanf to read in values from some sort of string (char array) data. The more modern and usable facility is to use std::stringstream. I like to focus on efficiency over everything else- so if that's not your bag I would suggest skipping this entry.

The inspiration for this blog entry came from "Exceptional C++ Style" by Herb Sutter, Item #2. I would strongly advise you to check it out if you want more detailed explanations about what I'm writing about. There's a cool chart in the article that shows the trade-off's between stringstream, boost::lexical_cast, and scanf. The claims made in the book are that scanf is more efficient because of the lack of one memory allocation call. This blog entry investigates this claim.

To understand each, let's write a typical program including both styles:

// Ahh, these make me feel good!
#include <iostream>
#include <sstream>
#include <string>
// Yikes, this makes me feel sick.
#include <cstdio>

enum { BUFFER_SIZE = 100 };

int main(int argc, char** argv)
{
    std::string input("Hello World!");
 
    std::string cpp_contents;
    std::stringstream ss;
    ss << input;
    ss >> cpp_contents;
     char c_contents[BUFFER_SIZE];
    sscanf( input.c_str(), "%s", c_contents);

    std::cout << "CPP Contents: " << cpp_contents << std::endl;
    std::cout << "C Contents: " << c_contents << std::endl;
}

So obviously this is skewing in terms of efficiency. But is there anything that can be done to improve the C++ side? Let's go line by line for the analysis:

std::input("Hello World!");

This line is not what we want to focus on. I used a string here, but we could easily replace it with a null terminated character buffer. In both cpp/c land we are taking its contents (.c_str()) and using that input.

std::string cpp_contents;
std::stringstream ss;

These two lines are just setup so far. Obviously the creation of the stringstream is a unique object so it's going to cost something. Also the cpp_contents are what we are going to read into.

ss << input;
ss >> cpp_contents;

So to fake things out we'll stream input into the string stream to populate it then stream a single string into cpp_contents. Thus far the overhead is the creation of two objects (cpp_contents, ss) and streaming in to populate the stringstream then streaming out to get the single string.

char c_contents[BUFFER_SIZE];
sscanf(input.c_str(), "%s", c_contents);

The analysis at this point doesn't seem fair because it's obvious that sscanf is more efficient because there is only a character buffer created. Character buffers make me cringe because they are fragile, prone to buffer overruns, and require more maintenance. sscanf does a very efficient job of filling a buffer. Can we make the C++ better?

The first thing I thought of was replacing the string cpp_contents with a character buffer. This is a stupid idea because all we would be doing is trading one object for another instead of getting rid of an object altogether.

The second thing I thought about was directly populating the stringstream. The stringstream object has a second constructor that takes a string! Therefore we can eliminate that first stream to populate the object. But our tally still stands at 2 objects and one operation for the C++ implementation, 1 object and 1 operation for the C implementation. How sad.

As a final test, I'm taking my two implementations and racing them:

#include <iostream>
#include <sstream>
#include <string>
#include <cstdio>

enum { BUFFER_SIZE = 100, MAX = 700000 };

//#define CPP_IMPLEMENTATION
#define C_IMPLEMENTATION

int main(int argc, char** argv)
{
    std::string input("Hello World!");

#if defined CPP_IMPLEMENTATION
    for(unsigned long i=0; i<MAX; ++i)
    {
        char cpp_contents[BUFFER_SIZE];
        std::stringstream ss(input);
        ss >> cpp_contents;
    }
#endif

#if defined C_IMPLEMENTATION

    for(unsigned long i=0; i<MAX; ++i)
    {
        char c_contents[BUFFER_SIZE];
        sscanf(input.c_str(), "%s", c_contents);   
    }

#endif

}

 

What I found:

CPP_IMPLEMENTATION
    real     0m0.547s
    user     0m0.544s
    sys      0m0.000s

C_IMPLEMENTATION
    real     0m0.062s
    user     0m0.056s
    sys      0m0.004s

There it is. If you are coding for safety, use C++'s modern standard string stream. However if you are focusing your endeavours on efficiency then seriously consider using a function from the scanf family. It's faster because of the obvious: no need for an extra object allocation.




Sunday, July 14, 2013

Exception Neutral vs. Exception Safe

The last entry I made was all about throw specifiers. Now I want to cover exception safety vs. exception neutral; the two main "degrees" of exception safety guarantees. The book Exceptional C++ has a (pardon the expression) exceptional explanation of these two levels- but not enough clarification in the examples. I thought I'd take the time to present some code examples that illustrate exception safe and exception neutral guarantees. 

Exception Neutrality

From what I understand (and I'll cite this), all exception neutrality is means that if you aren't handling an exception allow the exception to propagate [2]. So the following example would NOT be exception neutral:

void exceptionNonNeutral()
{
    try 
    {
         functionThatCouldThrow();
    }
    catch(...)
    {
        // Eat all exceptions thrown!
    }
}

I used to be a fan of eating all exceptions and throwing them away- however this doesn't allow exceptions to be handled by callers further down the stack. It can hide problems that may exist in functionThatCouldThrow(). To make this function exception neutral, might I suggest an extra throw?

void exceptionNeutral()
{
    try 
    {
         functionThatCouldThrow();
    }
    catch(...)
    {
        // Adding this line allows us to re-throw whatever was
        // thrown earlier. This is the key for exception 
        // neutrality.
        throw;
    }
}

The funny thing is that if you never added a try-catch block (i.e. no exceptions) you are automatically exception neutral because the exception would unwind the stack. It's only when you start catching exceptions that you need to worry about being neutral to exceptions that aren't understood by your function.

Exception Safe

This one is a bit harder to determine (and this is where Sutter scared me in Exceptional C++). Any function could throw an exception- which should make developers paranoid. This is also why writing exception safe code is so difficult as every single function could throw something and we have no idea what that might be! The only thing we can do as sane rational people is take care of our own code and safeguard it in case an exception is thrown.

The following is not exception safe:

#include <iostream>
#include <stdexcept>

struct A
{
    A(bool throwSomething) 
    { 
        if (throwSomething)
        {
            throw std::runtime_error("Exception!");
        }
    }

    int magicNumber() const { return 13; }

};

static A* internalObject = 0;

void reset()
{
     delete internalObject;
     internalObject = new A(true);
}

int getMagicNumber()
{
    return (0 != internalObject ? internalObject->magicNumber() : 0);
}

Let's cover what is going on. I've created a structure A which may or may not throw based on the bool passed in. The purpose is to illustrate exception safety- in the real world this could be a call to new that throws, etc. For the sake of simplicity, we'll keep it simple. 

The function reset() is not exception safe because it leaves the program in an inconsistent state. In a normal situation, getMagicNumber() would return a number with no incidents whether or not reset() was called. If getMagicNumber() calls without a valid internalObject it will return 0. However, if internalObject is non-zero and points at memory it will return the contents of the magicNumber() method.

Here's what can go wrong and why we should all be paranoid, if during reset() we invoke the constructor for A and it throws then the internalObject has already been deleted. That's really bad! The internalObject was destroyed, yet the internalObject pointer value is still non-zero. Then our user calls getMagicNumber() which  states that yes, internalObject is non-zero (but pointing at destroyed memory) and therefore we're going to call magicNumber() on that garbage memory. 

Exception safety means we have to keep the state of the program intact. Therefore we could do the following to fix this program:

#include <iostream>
#include <stdexcept>

struct A
{
    A(bool throwSomething) 
    { 
        if (throwSomething)
        {
            throw std::runtime_error("Exception!");
        }
    }

    int magicNumber() const { return 13; }

};

static A* internalObject = 0;

void reset()
{
    // Store a pointer to the old object.
    A* old = internalObject;

    // Attempt to create a new A object, if this throws
    // then internalObject is never set.
    internalObject = new A(true);

    // Delete the old object.
    delete old;
}

int getMagicNumber()
{
    return (0 != internalObject ? internalObject->magicNumber() : 0);
}

So here's the moral of the story, to make functions exception safe sometimes it's just a matter of ordering. In the fix above all that was done is to move the delete to the end. Delete operations should always be non-throwing [3]- that's the rule in C++. There's another operation that should also be non-throwing and that is "swap". A non-throwing swap is a good foundation to insuring exception safe code. The fix above is very close to using a swap operation. 



[3] - Exceptional C++, Item 16 - Herb Sutter

Exception Specifiers

In the past few years I've gotten away from using exceptions. There are a lot of reasons why- but I blame Sutter. His book "Exceptional C++" scared me for life from using exceptions. That was a few years ago so I figured it was time to face my fears about exceptions and given them another try. The first topic I want to explore is throw specifiers. 

While doing some research on the internet I found the following article from 2002 about throw specifiers [1]. The first thing read was specifying that a function does not throw anything, for example:

void doSomething() throw()
{
    // BLAH
}

This tells users that the doSomething() function shouldn't be throwing any exceptions. However, exceptions can still be thrown from doSomething() but the compiler won't do what you expect! For example, say you did the following:

void doSomething() throw()
{
    throw std::runtime_error("Exception!");
}

And then you called it- what ends up happening with g++ 4.7 is that the runtime_error gets thrown and the program crashes with: 

terminate called after throwing an instance of 'std::runtime_error'
  what(): Exception!
Aborted (coredumped)

Okay, so throw specifiers can specify that no exceptions should be thrown from a function but yet the functions can still be thrown? What kind of backwards feature is this? There's more to the confusion, try catching the exception in this form using this program:

#include <iostream>
#include <stdexcept>

void doSomething() throw()
{
    throw std::runtime_error("Exception!");
}

int main(int argc, char** argv)
{
    try 
    {
        doSomething();
    }
    catch(...)
    {
        std::cout << "CAUGHT IT!" << std::endl;
    }
}

The ellipses (...) in the catch is a catch-all. Any exceptions thrown will be caught and discarded in this program. Here's the unusual part: when a throw specification is added and an exception is thrown that is NOT specified it appears as though the program will not allow a catch in any form and it becomes an automatic crash/terminate! That's really confusing- but it's a case of exceptions behaving the way exceptions should and NOT how the programmer expects.

I guess this is just one of the strange quirks I find scary about using exceptions. To allow a runtime exception to escape you can modify the doSomething throw specification to look like:

#include <iostream>
#include <stdexcept>

void doSomething() throw(std::runtime_error)
{
    throw std::runtime_error("Exception!");
}

int main(int argc, char** argv)
{
    try 
    {
        doSomething();
    }
    catch(std::runtime_error const& err)
    {
        std::cout << "CAUGHT IT!" << std::endl;
    }
}

I also modified the catch to catch a specific exception- and notice it's done by reference. The rule of thumb is to "throw by value, catch by reference" [2]. 

REFERENCES


[2] - Effective C++, Scott Meyers. Item 13