Where to find Qt Configure Flags

Introduction

It's Sunday afternoon and I'm about to re-setup a development virtual machine. In the creation of it I'm going to be building Qt5 from source on Linux. Building from source can be troublesome at times because it's hard to find good information about the configure "script". This small entry is about where to find information (from source code) about Qt's configure "script".

If I do a quick google search with "Qt configure documentation", the first good link I get is for Qt4:

http://qt-project.org/doc/qt-4.8/configure-options.html

Not bad. The link below it is for Qt 5.4:

http://doc.qt.io/qt-5/configure-options.html

In the tradition of ever obtrusive documentation, it's very wordy and hard to find what you are looking for. In my opinion the documentation style for Qt5 is inferior to Qt4 (opinions are subject to change).

This entry is all about the actual mechanism behind the configure "script". I am putting script in quotation marks because it's not actually a script- it's an executable. The first step after pulling Qt down from source is to run "configure" in the root directory. Obviously if using Visual Studio you must have run this from the VS Command Prompt.

QT/configure [.bat]

There are two scripts in two languages named "configure" in the root directory. The regular old configure script is a shell script that simply executes qtbase/configure. The configure.bat does the same in its own way executing qtbase/configure.bat.

QT/qtbase/configure.bat

This script begins the "bootstrapping configure". Essentially it runs a perl script called syncqt.pl and then tries to build the configure tool in QT/tools/configure. The configure executable tool takes the place of the QT/qtbase/configure. When I built Qt 5.3 it looks as if this tool was never built- looks as if this is a Windows only step.

QT/qtbase/configure

This is a shell script that does the actual setup for the build. When using Visual Studio, this won't be utilized. Instead the configure application listed above is used. The reason why it exists this way is probably due to batch files being rotten but I could be wrong.

Where are the build options for configure?

In conclusion, to find the base options for the configure step in Qt the developer could look in QT/qtbase/configure (which is the script used for Linux) OR look qt QT/qtbase/tools/configureapp.cpp in the method Configure::parseCmdLine().

 - Visual Studio     - SOURCE : $QTDIR/qtbase/tools/configure/configureapp.cpp
   - TYPE   : Binary Application
   - METHOD : Configure::parseCmdLine()
 - Linux (Other)
   - SOURCE : $QTDIR/qtbase/configure
   - TYPE   : Shell Script
   - METHOD : Line 828? Look for "parse command line arguments"

Using the source code the developer can get definitive information about the configure arguments that can be passed into building Qt.

QQuickItem (Cannot assign object to list)

Introduction

I find myself with a problem that the Qt forums can't solve. The only way to fix it is to dive into the Qt source code and figure it out myself. This blog post is all about the internals of QtQuick2 and is meant for an intermediate audience. I apologize about this blogpost as it's mainly about how I'm fixing this one issue; but I assure the reader that once finished they'll have more intimate knowledge of QtQuick2.

Prerequisites for this entry include basic knowledge about QQuickItem and customization using C++. Before writing this article I was familiar with the class predecessor, QDeclarativeItem. The two are very similar yet very very different. That might be the subject of another post.

The first section is all about detailing the crime scene or in other words describing the problem. The second section documents some useful information about import's in Qt5/QtQuick2 (including some really general information about qmldir). The third section after that documents the "default property" in QML and what it is all about.The fourth section is all about where and how the definitions of QQuickItems are stored.

Problem Description

During a port of QtQuick1 to QtQuick2 code I got the error:

types.qml: 48:1: Cannot assign object to list

We've all been there- a problem we're seeing in a third party library. This is a kind of problem that no one else sees. We didn't want to deal with branching our code for the two distributions. Using some typedefs along with a hand rolled preprocessor the whole thing works great. Except when I saw the following error when loading some QML:

types.qml: 48:1: Cannot assign object to list

This was a strange error indeed. My QML program looked like this:

import QtQuick 2.0
import blah 1.0

TypesCreator {
    id: root
    ...

    Connections { // Line 48
        target: __start__ // __start__ is an object I bound a QObject to.
        onStart: {
            // Do something.
        }
    }
}

The TypesCreator element is part of the blah import that is custom to the application. In the interest of treating this article as a "whodunnit" type of story you can assume that TypesCreator's object looks like this in C++:

class TypesCreator : public QQuickItem
{
    Q_OBJECT
    public:

    private:
};

This obviously does nothing except acts as a container for other objects. In any crime scene you have to know the area. In this case I needed to know what was imported and what wasn't.

Where/What Modules Are/Exist?

The first theory I had was that the Connections object wasn't registered. Eventually it was read that Connections was technically not part of QtQuick 2 but rather was part of QtQml [1]. Upon further reading, QtQml components are automatically included as part of QtQuick. This theory is a dead end- however it led me to trying to solve the riddle of "what objects belong to what import"?

The answer is:

http://doc.qt.io/qt-5/modules-qml.html

As a sidenote to this section: the Qt5 documentation is quite irritating. While being much more comprehensible it fails to be searchable. It took me over 5 minutes to find the module list.

The module source code for the imports is organized under:

qtdeclarative/src/imports/

The easiest way to decipher the imports is to go under a directory and read the qmldir file. The qmldir file has all of the information about what types of classes for QML the module contains. For example I'm looking at:

qtdeclarative/src/imports/qtquick2/qmldir

This is what it reads:

module QtQuick
plugin qtquick2plugin
classname QtQuick2Plugin
typeinfo plugins.qmltypes
designersupported

Essentially it is a hacked together file format that attaches some extra information onto a module. In this case the first line defines what the module name is. The second line tells me that this is a collection of C++ objects that make up this module. The rest I'll skip for now- I just want to know where the basic information is at this point.

Default QML Properties

One of the biggest questions I had when trying to solve this problem was what property could this not assign an object to? I put that in bold mainly to get back to the original error message:

types.qml: 48:1: Cannot assign object to list

So what was this property? The only way I could actually solve this was to get a breakpoint into the code that was printing the error message and then work backwards. To achieve this purpose I:

1. Performed a grep to find the string "Cannot assign object to list" in the Qt Source code. It turned up in qtdeclarative/src/qml/compiler/qqmltypecompiler.cpp.
2. When I found the message I noticed that it passed in a propertyName as a parameter to the function, I outputted this using qDebug(). 
3. Finally to really throw a wrench into things I throw a standard exception. This allows Visual Studio to trip up each time it encounters this error.

When running I discovered the property name being set was "data". This was confusing since no where in my code did a "data" object exist. That's not true though! Inside of the source code for QQuickItem it defines the property "data" as:

Q_PRIVATE_PROPERTY(QQuickItem::d_func(), QQmlListProperty<QObject> data READ data DESIGNABLE false)

So every item in QML has a data property and apparently it's a list of QObject's. That's something new- however I'm never using the "data" property. As I was about to find out, this is also not true.

There's a feature in QML where the developer can specify the "default property" [3]. Inside of QQuickItem it has the following macro also added in:

Q_CLASSINFO("DefaultProperty", "data")

This means that any sort of unbound object created will be added to "data". Since every QML object has the default property set to "data" all of what would appear to be the child items are added in via the data property. It is mentioned in the documentation that the data elements when added are also added to the children list.

This explains a lot; it explains why the property it cannot add Connections to is called "data", it explains why the type it is trying to add to is referenced as a list. The only thing it doesn't solve is why Connections isn't registered as a QObject. It should be; something isn't adding up. The next logical step is to observe how the Connections object is added to a QML Engine.

The How and Where QML Types Are Stored

The final bit of the puzzle for this entry is determining why the QML types cannot be found. This involved stepping into Qt. I found out the following:

1) For each type defined for QML, there is a QQmlType object. According to my notes the declaration can be found in qqmlmetatype_p.h. [4]
2) There's an object of class QQmlMetaTypeData that contains a map of all the created types. When the developer calls qmlRegisterType, the type created, its name, and version information all end up in a map here.The declaration is found in qqmlmetatype.cpp [5].
3) The map itself holds a hash of type identifier to QQmlType (line 76 in the source).

My best guess is that the type identifier for a well known type is getting overwritten in this map. The basic type I'm trying to use was replaced with some garbage type. To understand this, I tracked down where the QML numeric type was defined. If walking up the call stack, the QML Type identifier is found to map to the QMetaTypeIdHelper<T>::qt_metatype_id() where T is the type of the class.

The summary to this section is that any time a registration occurs with the meta type system the identifier that is produced is also used in QML to map type information. The numbers are mostly guaranteed to be unique since they are using a basic atomic integer mechanism so it makes sense. Having this information in hand I could now surmise that somewhere I was doing something stupid when registering types. This guess proved to be right since I found this little gremlin in the code:

qRegisterMetaType<QObject*>("LayerItem");

Looking at it now I'm kicking myself. This was using the type id from QObject* to map to the string "LayerItem".

Conclusion

While searching for why the runtime error "Cannot assign object to list" in QML I learned a lot about how the language parses, interprets, and stores its programs:

• Modules in QtQuick2 are very similar to QtQuick1 modules with the exception they can store more meta information in qmldir.
• QObject based classes have the ability to store a default parameter which can be used to store "child" information in a custom way.
• QML Types are stored in a map inside of QQmlMetaTypeData. The id's used in the map come from the Qt Meta Object System.

In the process of learning about all of these facets of the engine I found the solution to my problem which was to get rid of a call to qRegisterMetaType<QObject*>. 

REFERENCES

[1] - http://doc.qt.io/qt-5/qtqml-qmlmodule.html

[2] - http://qt-project.org/doc/qt-4.8/declarative-cppextensions-referenceexamples-default.html

[3] - http://doc.qt.io/qt-5/qtqml-syntax-objectattributes.html#property-attributes

[4] - https://qt.gitorious.org/qt/qtdeclarative/source/ccd69247eaa6289bc31115f863ba4737b14fe34e:src/qml/qml/qqmlmetatype_p.h

[5] - https://qt.gitorious.org/qt/qtdeclarative/source/e2ea0a83cc876fb54a2a8bf6f1350dbfa52f596b:src/qml/qml/qqmlmetatype.cpp

Qt 5 + Android (January 2014 Edition)

Introduction (2014.01.02)

Sorry this is so late- as usual life gets busy. The last entry I made about Qt5 and Android ended with a stuttering failure. This entry focuses on Linux and building Qt 5 modules for Linux. There are instructions for this at http://qt-project.org/wiki/Qt5ForAndroidBuilding but the instructions assume you know a lot about Android/Qt before starting. This guide is aimed at the semi-knowledgeable that are partially in the know about Qt5 and Android technologies.

The architecture I'm using is Linux x86_64 so please take that into account when following the instructions. Let's get started:

Step 1: Download Android SDK/NDK

For steps that involve downloading from http, I fallback to a tool called wget. You can also scour the internet and find wget for Windows- it's a handy tool to have around when scripting.

# https://developer.android.com/sdk/index.html
wget http://dl.google.com/android/adt/adt-bundle-linux-x86_64-20131030.zip

# http://code.google.com/p/mingw-and-ndk/downloads/list
wget https://mingw-and-ndk.googlecode.com/files/android-ndk-r8e-ma-linux-x86_64.tar.xz

Obviously if you are working with a 32-bit machine use the x86 binaries. Once these files are downloaded and placed somewhere (let's say locally into /home/john/sdk and /home/john/ndk) we can move on. FYI, to extract a tar.xz file in Linux use:

# Saving you a google.
tar -xJf android-ndk-r8e-ma-linux-x86_64.tar.xz

There's a step here that isn't really elaborated upon in the main instructions: installing an Android API. They casually have mentioned you will probably want API's 10, 11, and 16. There's a problem, we are command-line cool so we don't want to start a user interface just to pull these down. No problem! Do the following:

cd /home/john/sdk/tools
./android update sdk -u --filter android-10,android-11,android-16

It will prompt you to continue with a "y". If you want to be really cool, add a file in the same location called "y.txt" that contains the letter y and a newline. Save it. Then when you would do the android update, do this:

./android update sdk -u --filter android-10,android-11,android-16 < y.txt

Redirect that input into the updater. Perfect! License accepted. Say you don't want to do all of this by hand, just check out my script for installing the adt-bundle.pl (for linux-x86_64 only) at https://github.com/signatal/system/blob/master/tools/adt-bundle.pl . (PS - If you're looking to help out on the automation side let me know).

You must also set the following environment variables:

ANDROID_SDK_ROOT : Obviously set this to your sdk root that was downloaded.

ANDROID_API_VERSION : android-16 is what I have now started setting this too (see notes in Step 6 as to why this might be an issue). These environment variables MUST be present before configuring Qt to be built! Why these aren't configure command line options? Who knows, maybe they are?

Step 2: Familiarize Yourself with the NDK

The NDK you've downloaded for Android contains some prebuilt cross-compilers that are helpful when building Qt5. If you want the binaries to work on an Android system they need to be built with one of these compilers. To locate one of the compilers, navigate into the ndk toolchains directory:

cd /home/john/ndk
cd toolchains

# The following is the name of one of our selected cross compiler
# selections.
cd arm-linux-androideabi-4.7

cd prebuilt/linux-x86_64

# Just take a look at what's here.
ls

Notice that this feels surprisingly like a linux /usr directory. It's complete with include, bin, lib, share, etc. directories. It's almost as if someone set this up so that a single environment variable could point at it to build Android binaries. In the bin directory you won't see the normal gcc/g++: instead you'll see programs prefixed with "arm-linux-androideabi-".

There's another interesting directory here called arm-linux-androideabi which is at the same directory and include/share/bin/lib. If you traverse this one and go into another bin directory you'll find the C++ compilers and tools.

At this point I had to ask the question: I know what arm, linux, and android are. But what does ABI stand for? It stands for Application Binary Interface- and it's the means by which the Operating System knows how to execute an application. This compiler will produce code that will run on an Android operating system.

Step 3: Obtain Qt5

Most tutorials will actually set the environment up as you go. Going to take a reverse approach and download Qt and try to build it without the environment variables. The reason is because I want to be sure that what I am writing is correct and I want to document what happens when the environment variables aren't right.

# Clone Qt and place it into a directory called qt5. You might
# think, "Oh, this is surprisingly fast!"
git clone git://gitorious.org/qt/qt5.git qt5

cd qt5

# You might think, "Oh, this is suprisingly slow!" Clones each
# module individually so takes awhile.
perl init-repository

Step 4: Strike Gold

You want to know why I split up Obtaining Qt5 and Building Qt5? The reason is I want to explain what the configure scripts are going to look for when building using the Android compiler. Navigate into qt5/qtbase. Then open the script "configure". Just do it- it's not that bad.

cd qt5
cd qtbase
vim configure

Look for the line in the file that looks kind of like:

QMAKE_CONF_COMPILER=$CFG_DEFAULT_ANDROID_NDK_ROOT/toolchains/$ANDROID_NDK_TOOLS_PREFIX-$CFG_DEFAULT_ANDROID_NDK_TOOLCHAIN_VERSION/prebuilt/$CFG_DEFAULT_ANDROID_NDK_HOST/bin/$ANDROID_NDK_TOOLS_PREFIX-g++

Whew. That's a big line! But there's a method to my madness. This line is pure gold for us trying to build this damn thing because in one line it's giving us a hint as to what the input variables should look like. It's having an equation and knowing the answer, we just have to solve for the unknowns!

$CFG_DEFAULT_ANDROID_NDK_ROOT : We know this to be /home/john/ndk/android-ndk-r8e. Now we go into /home/john/ndk/android-ndk-r8e/toolchains to see what is there (the answer is all sorts of stuff).

$ANDROID_NDK_TOOLS_PREFIX : This could be arm, llvm, mipsel, x86. Most likely our default we are building for will be arm (most Android architectures are arm-based). This can be respecified if you want to by using the -android-arch flag, just stick with armeabi-v7a for now though! When that's the case, the prefix used will be arm-linux-androideabi. The platform architecture will be arch-arm.

$CFG_DEFAULT_ANDROID_NDK_TOOLCHAIN_VERSION : From above, the only values this could be are 4.4.3, 4.6, 4.7.

$CFG_DEFAULT_ANDROID_NDK_HOST : This must be linux-x86_64.

After putting it all together, we can determine that the compiler that will be used is:

/home/john/ndk/android-ndk-r8e/toolchains/arm-linux-androideabi-4.7/prebuilt/linux-x86_64/bin/arm-linux-androideabi-g++

Step 5: Build Qt5

I put a great deal of time in pointing out various parts of the NDK for a reason. In the above steps we've already downloaded the Android SDK/NDK and pulled down and initialized the Qt5 project. We've polked around at the contents of the NDK to kind of get the feel about what might be necessary to build Qt 5 for Android. Like most big libraries we need to run a configure step before we can build. Qt is no exception- especially when it comes to building with a cross-compiler.

The configure step I suggest using is:

./configure -developer-build -opensource -confirm-license -xplatform android-g++ -nomake tests -nomake examples -android-ndk /home/john/ndk -android-sdk /home/john/sdk -android-ndk-host linux-x86_64 -android-toolchain-version 4.7 -skip qttranslations -skip  qtwebkit -skip qtserialport -skip qtwebkit-examples -no-warnings-are-errors

Let me break this down:

./configure : Obviously this is the configure script invocation. Everything following it is just command-line options.

-developer-build : Use this flag if you don't plan on performing the installation step.

-opensource : This flag will automatically choose the "Open Source" version of Qt. You can leave this out if you want, but it will prompt you for Open Source/Commercial. Just figured I'd help you speed things along.

-confirm-license : Same as Open Source, this will confirm the license automatically. It can also be not added, but it will also stop configuration to ask.

-xplatform android-g++ : If you look this up in the configure script, it states that this will set "The target platform when cross-compiling" (the x is for cross- just in case you didn't get it).

-nomake tests -nomake examples : Don't make these things, we don't need em.

-android-ndk /home/john/ndk : Specify where the ndk is located. This is a CRUCIAL step! The reason it's crucial is that we get our compiler tools and environment from this location.

-android-sdk /home/john/sdk : Specify where the sdk is located. This is also a crucial step.

-android-ndk-host linux-x86_64 : Specify your host system. If you are in doubt about the exact value of this string, just look at my section above to try and determine this for yourself. Never hurts to double check.

-android-toolchain-version 4.7 : Aren't you glad I made you go through the NDK directories? Choose a version string inside of your NDK/toolchains directory that makes sense. In my case, I had versions for 4.4.3, 4.6, and 4.7 given my host system. The Qt configure script uses this toolchain version to help locate the compiler tools/environment, so choose wisely.

-skip qttranslations -skip  qtwebkit -skip qtserialport -skip qtwebkit-examples -no-warnings-are-errors : Same skippy stuff as usual.

Whew. Breaking that down was a chore and a half- so I hope it saves someone a few minutes of time later. There is more to this story! Even if you do somehow configure the project correctly it can still bail out on you early because of the absence of some magical environment variables.

Step 6: Problems

After compiling I had a really strange error occur with some Java code:

javac -source 6 -target 6 -Xlint:unchecked -bootclasspath /home/jr/system/tools/android-sdk/adt-bundle/sdk/platforms/android-10/android.jar -cp /home/jr/system/tools/qt5-android-build/qt5/qtbase/src/android/accessibility/jar/src/:/home/jr/system/tools/android-sdk/adt-bundle/sdk/platforms/android-10/android.jar -d .classes.bundled src/org/qtproject/qt5/android/accessibility/QtAccessibilityDelegate.java src/org/qtproject/qt5/android/accessibility/QtNativeAccessibility.java
src/org/qtproject/qt5/android/accessibility/QtAccessibilityDelegate.java:62: error: cannot find symbol
public class QtAccessibilityDelegate extends View.AccessibilityDelegate
                                                 ^
  symbol:   class AccessibilityDelegate
  location: class View
src/org/qtproject/qt5/android/accessibility/QtAccessibilityDelegate.java:99: error: cannot find symbol
    public AccessibilityNodeProvider getAccessibilityNodeProvider(View host)
           ^
  symbol:   class AccessibilityNodeProvider
  location: class QtAccessibilityDelegate
src/org/qtproject/qt5/android/accessibility/QtAccessibilityDelegate.java:193: error: cannot find symbol
    private AccessibilityNodeInfo getNodeForView()
...

I know this doesn't normally fit this blog (due to it being a C++ technologies blog), but I want to try and fix this. Without this working it appears as though both Declarative and Quick modules will not build- leaving us with half-baked Qt Libraries. There was something in the Qt documentation about android-10 being the earliest API that could be used. I believe this to be incorrect now- since the error above is complaining about not finding View.AccessibilityDelegate, which was just introduced in API-14 of the Android SDK:

http://developer.android.com/reference/android/view/View.AccessibilityDelegate.html

This leads me to my final confusion: all blog posts suck because they fall quickly out of date. I'm sure if five minutes have passed since I hit Post that this entry will also be chalked full of bad information. I encourage everyone who reads this to PLEASE comment whether or not these instructions worked for you- it will help everyone in the future who happens to stumble in.