Jutting Bytes

Digressions of a research engineer

Fixing the Qt Mysql Driver on MacOSX 10.11

| Comments

A vanilla intall of Qt, using either their online or offline installers, comes with sql plugins. Their however depend on libraries that may not be installed on your system.

I came with this issue trying to use the QMYSQL driver.

1
2
3
✓ jwintz@inextremis ~/Development/112m-server/build $ ./bin/112m-server
QSqlDatabase: QMYSQL driver not loaded
QSqlDatabase: available drivers: QSQLITE QMYSQL QMYSQL3 QODBC QODBC3 QPSQL QPSQL7

So I checkout out the dependencies of the plugin qsqlmysql:

1
2
3
4
5
6
7
8
9
10
11
✓ jwintz@inextremis ~/Development/qt/5.5/clang_64/plugins/sqldrivers $ otool -L libqsqlmysql.dylib
libqsqlmysql.dylib:
  libqsqlmysql.dylib (compatibility version 0.0.0, current version 0.0.0)
  /opt/local/lib/mysql55/mysql/libmysqlclient.18.dylib (compatibility version 18.0.0, current version 18.0.0)
  /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
  @rpath/QtSql.framework/Versions/5/QtSql (compatibility version 5.5.0, current version 5.5.1)
  @rpath/QtCore.framework/Versions/5/QtCore (compatibility version 5.5.0, current version 5.5.1)
  /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration (compatibility version 1.0.0, current version 1.0.0)
  /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
  /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
  /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)

Note the dependency /opt/local/lib/mysql55/mysql/libmysqlclient.18.dylib which assumes that one has mysql installed using either Fink or Macports most probably. So if you don’t have or want to use Macports, you’ll have to patch the library to use another instance of the libmysqlclient.

For many reasons, I personally prefer Homebrew, which is a very smart tool (try its doctor option for example).

1
✓ jwintz@inextremis ~/Development/112m-server/build $ brew install mysql

Then, locate your libmysqclient.dylib path.

1
2
3
4
✓ jwintz@inextremis /usr/local/Cellar $ find . | grep libmysqlclient
./mysql/5.7.10/lib/libmysqlclient.20.dylib
./mysql/5.7.10/lib/libmysqlclient.a
./mysql/5.7.10/lib/libmysqlclient.dylib

The last step is then to patch the qt mysql plugin to use this newly installed instance.

1
✗ jwintz@inextremis ~/Development/qt/5.5/clang_64/plugins/sqldrivers $ install_name_tool -change /opt/local/lib/mysql55/mysql/libmysqlclient.18.dylib /usr/local/Cellar/mysql/5.7.10/lib/libmysqlclient.dylib libqsqlmysql.dylib

Enjoy and save some time !

Using Clang-llvm Compilation Databases

| Comments

Compilation databases are JSON files that are formatted so as to specify how to replay single compilations independently of the build system.

It consists of “command objects”, where each entry specifies one way a translation unit is compiled in the project, that is: a working directory, a command including the compiler name with its include directories and definitions options and flags, as well as the file for which the object has been generated.

As stated on clang’s web site, tools based on the C++ Abstract Syntax Tree (AST) need full information how to parse a translation unit. This information is usually contained in various ways in the build system of a project.

The problem is that relaying only on such tools is not optimal as they are change driven and that figuring out wether things have changed is often an IO bound process.

But what are they useful ? Well, it is very efficient if you’re interested in source code writing analysis and source code rewriting tools. As an example, I’m using it for both code completion (using company) and linting (using flycheck) with emacs.

But how does one generate such compilation databases ? Well there is three ways I know of.

  • If using cmake, simply pass the -DCMAKE_EXPORT_COMPILE_COMMANDS=ON
  • If using ninja, invoke ninja -t compdb
  • Otherwise, when ending up with Makefile, use bear make

Bear, which stands for Build EAR executes original build commands and intercepts the exec calls issued by the build tool. This is achieved using code injection.

On Preload and SIP on MacOS X.11

| Comments

I’ve always been particularily fond of the PRELOAD trick that enables one to override symbols when launching an executable.

On linux, invoke your binary as such:

1
$ LD_PRELOAD=somelib.so executable

On MacOSX, invoke your binary as such:

1
$ DYLD_FORCE_FLAT_NAMESPACE=1 DYLD_INSERT_LIBRARIES=somelib.dylib executable

A classical use case is turning a non stereographic OpenGL implementation of a program into a quad-buffer stereographic one by redefining the calls to some gl functions.

This is also used by the wonderful bear tool, that catches commands corresponding to Makefile rules in order to create a compilation database, that is useful e.g. for intellisense.

However, Apple, with OSX El Capitan (10.11), has introduced System Integrity Protection (SIP), that prevents this trick from working, silently. In the meantime, it is no more possible to use tools such as dtrace or dtruss.

Although not recommended by Apple, one can entirely disable SIP on a machine, by booting in recovery mode and invoke the following command.

1
$ csrutil disable

Preloading will then work, at the detriment of security. Finally, I found this excellent post which explains that SIP can be partially disabled.

1
2
3
4
5
$ csrutil enable --without kext
$ csrutil enable --without fs
$ csrutil enable --without debug
$ csrutil enable --without dtrace
$ csrutil enable --without nvram

I have not tested which combination is minimal and sufficiant in order to allow preloading, if someone knows, please put a comment.

Faster Cursor Movement in Emacs

| Comments

While hacking I like to get my ideas come true on the screen as quick as they come. This is why I have always wanted to learn “touch-typing” on the one hand, and why I am so concerned about my hacking tools and their setup on the other hand. The toolchain contains Emacs, of course.

While digging mouse movements in the emacs lisp documentation, I realized there was not much functions or variables to adjust, Emacs’ key movement is actually driven by the host operating system in terms of key repeat delay.

So I went to my favorite’s OSX System preference pane, and looked for Keyboard, here is what I found.

I moved the key repeat delay to its fastest possible value and also set the timer before the key repetition is toggled to its shortest value, and now, I am a happy coder that can very quickly edit my code.

Note that this setting has the benefit to affect all keyboard interaction with your system.

Fixing the Brew Linkapps Issue on MacOSX

| Comments

Today, I have been extensively working on my emacs prolusion and updated emacs to its current HEAD version, that is, 25.1.50.1.

To do so, I have obviously used brew:

1
$ brew reinstall emacs --use-git-head --cocoa

This builds Emacs.app into brew’s cellar, and to have it into your /Applications folder, brew invites you to invoke the following command:

1
$ brew linkapps emacs

This does the job, it symlinks Emacs.app from brew’s cellar to the Applications folder but it appears that symlinks are not supported by Spotlight.

To fix this, I found a nice one-liner solution. First remove the symlink:

1
$ rm /Applications/Emacs.app

Then:

1
$ osascript -e 'tell application "Finder" to make alias file to POSIX file "/usr/local/opt/emacs/Emacs.app" at POSIX file "/Applications"'

Spotlight will now index Emacs.app. The same procedure applies for other application bundles provided by brew such as MacVim.app.

Helm-Dash Makes You Efficient

| Comments

Every coder will say so: when learning a new language or using a new toolkit or library, a great documentation makes the difference. It is however usually really boring to browse when it comes as a pdf, or in html format.

Documentation engines, including doxygen, qdoc, appledoc among many great others, provide indexation and allow their authors to generate databases for fast queries of symbols. This is where IDEs come, as the documentation lookup becomes part of the code writing experience.

They however present many disadvantages: they are usually bound to a language and a small set of libraries. Also, they hide many aspects of the development life cycle: managing dependencies, hiding linking and post build phases. I won’t argue on why these are drawbacks, but ail those reasons make me use a single editor for all languages I’m coding with, all tools I’m using in a development cycle: Emacs.

I’ve been using macs for many years now, but I happen to hack on other operating systems as well. There is one software that made me love coding on the mac: Dash. It is an API documentation browser that pops up with a key sequence and disappears when it looses focus. It hosts and downloads more than 130 famous docsets and provide very fast searching and a nice browsing experience. Also, its author, Bogdan Popescu is a very nice person, easy to talk with and is very open when it comes to contributing to Dash, see this post.

Helm is an emacs incremental completion and selection narrowing framework, a fork of the well known anything.el, that is currently very popular for many developers are using it as a frontend for their own packages.

Here comes Helm-Dash, a wonderful emacs packages, available on Melpa that uses Dash docsets inside emacs to browse documentation. Note that is does not require the Dash app, and so, works on any platform, just as Emacs !

QChart.js: QML Bindings for Chart.js

| Comments

Working with Qt/QML on a new project on electro-magnetism data collection, I wanted to implement some simple yet meaningful and eye friendly charts to display data. It happens that some days before, I used Chart.js on the web backend of this mobile application and thought that for the sake of consistence I would give it a shot with QML since it integrates nicely with javascript.

Find it on GitHub.

QMake: The Maverick’s Case

| Comments

I’ve just spent hours figuring why after upgrading my Xcode to version 5, building projects using cmake goes fine while I have lots of undefined symbols using qmake. Let’s take an example.

1
2
3
4
5
6
7
8
9
10
Undefined symbols for architecture x86_64:
  "___sincos_stret", referenced from:
      _q_make in libquat.a(quat.c.o)
      _q_from_axis_angle in libquat.a(quat.c.o)
      _q_from_two_vecs in libquat.a(quat.c.o)
      _q_from_euler in libquat.a(quat.c.o)
      _q_euler_to_col_matrix in libquat.a(matrix.c.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [bin/is-calibrator] Error 1

sincos_stret looks very much like a libmath symbol, even though not defined.

1
$ nm /usr/lib/libm.dylib | c++filt | grep sincos_stret

Clang-LLVM performs system optimization, so with this very new version of Xcode, many math functions end up in libsystem_m.

1
2
$ nm /usr/lib/system/libsystem_m.dylib | c++filt | grep sincos_stret
    0000000000014143 T ___sincos_stret

However using 10.8 SDK, the latter does not contain our symbol.

1
nm /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/lib/system/libsystem_m.dylib | c++filt | grep sincos_stret

But on 10.9 SDK, here it is.

1
2
nm /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/lib/system/libsystem_m.dylib | c++filt | grep sincos_stret
    0000000000014143 T ___sincos_stret

So why don’t we touch it when linking ? Well, here is the reason. Using qmake, (whatever the version of qt, tried with 5.1.x, 5.2.x), qmake’s macx-clang spec uses MacOS 10.8 SDK. Here comes qmake -d, a nice option logging what happens behind the scenes.

1
DEBUG 1: QMAKE_MAC_SDK.macosx.path := /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdK

We want to use 10.9, so that linking on math gives us our symbol definition. Overriding qmake’s mac sdk is not something you easily find in the (however brilliant) qt docsets. If the debug output gives us some hints, we also want to set the deployment target flag. The project file snippet is as follows.

1
2
macx:QMAKE_MAC_SDK.macosx.path = /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdK
macx:QMAKE_MACOSX_DEPLOYMENT_TARGET = 10.9

Enjoy and save some time!

Qt on iOS Examples

| Comments

The Qt project has recently published a post on recent advances in writing an iOS platform plugin. So far, many features are working just as expected, and I must say, these features cover the needs of many many applications we have written so far. So yes, we can move forward porting our applications to iOS.

The first part of this post shows how to set up a development environment, note it merely summarises information given on the post, plus some additional hints to help you run the examples provided in the second part of this post.

The second part introduces a repository that demonstrates the behaviour of some relevant features of Qt, both in desktop, simulator and device configurations. Note they do no constitute a full demo application (check this one if you are looking for one), but a set of barely enhanced feature based hello world programs.

As usual, repository on GitHub.

QLeap

| Comments

Leap represents an entirely new way to interact with your computers. It’s more accurate than a mouse, as reliable as a keyboard and more sensitive than a touchscreen. For the first time, you can control a computer in three dimensions with your natural hand and finger movements.

Late october, the following post has been published on LEAP’s blog, inviting people to register to the early admittance program, including a device and an access to LEAP’s SDK. LEAP motion is a really interesting innovative product. We wanted to build wrappers to let anyone using Qt and/or Qml to be able to interact with one of those.

We are pleased to let you know that our subscription has been selected. We already have some interesting results, but I’ve decided to create a dedicated project on GitHub – http://github.com/jwintz/qleap – that we will rewrite with Qt5, and populate as development goes on. A preliminary page that introduces the idea has been setup here: http://jwintz.me/qleap/.